Camera calibrating apparatus

ABSTRACT

Herein disclosed is a camera calibrating apparatus for calibrating an optical parameter of a camera mounted on an automotive vehicle, comprising: a first housing position information storing section ( 115 ) for storing first housing position information, a second housing position information storing section ( 116 ) for storing second housing position information, a first optical position information producing section ( 117 ) for producing first optical position information, a first optical position information storing section ( 118 ) for storing the first optical position information, a second optical position information producing section ( 120 ) for producing second optical position information, a second optical position information storing section ( 130 ) for storing the second optical position information, and a calibrating section ( 160 ) for calibrating the second optical position information stored in the second optical position information storing section ( 130 ) on the basis of image information obtained by a camera ( 110 ) in a second coordination system ( 102 ).

TECHNICAL FIELD OF THE INVENTION

The present invention relates to a camera calibrating apparatus, andmore particularly to a camera calibrating apparatus for calibrating acamera mounted on an automotive vehicle or the like.

DESCRIPTION OF THE RELATED ART

In recent years, there has been developed an imaging controllingapparatus, for example an electronic control unit (herein after simplyreferred to as “ECU”), which is used to be electrically connected to acamera mounted on an exterior surface of an automotive vehicle to detectthe position occupied by an object located on a road, in particular, inthe vicinity of the automotive vehicle on the basis of image informationobtained by the camera. When, in general, the camera is mounted on theautomotive vehicle in combination with the imaging controlling apparatusof this type, the calibration of the camera is performed by a cameracalibrating apparatus in order to specify optical parameters of thecamera.

Up until now, there have been proposed a wide variety of cameracalibrating apparatuses of this type, all of which are classified intotwo different types including a first type to calibrate the camerabefore that camera is mounted on the automotive vehicle, and a secondtype to calibrate the camera after that camera is mounted on theautomotive vehicle. For the purpose of facilitating an assembly work ofthe camera to the automotive vehicle, there is a growing demand for thecamera calibrating apparatus for calibrating the camera mounted on theautomotive vehicle in recent years.

As shown in FIGS. 39 to 41, the conventional camera calibratingapparatus 500 is electrically connected to the camera 510 as an imagingdevice. The camera 510 includes a housing unit 511 and an opticalsection 512 supported by the housing unit 511 to obtain imageinformation through the optical section 512.

The conventional camera calibrating apparatus 500 comprises a firsthousing position information storing section 515 for storing firsthousing position information of the housing unit 551 obtained on thebasis of the first coordinate system 501 defined in a camera producingplant, and a second housing position information storing section 516 forstoring second housing position information of the housing unit 551obtained on the basis of the second coordinate system 502 defined in anautomotive vehicle producing plant.

The camera calibrating apparatus 500 is adapted to revise the camera 510in the camera producing plant by using a revising marker 505 located inthe first coordinate system 501. Here, the term “the calibration of thecamera 510” is intended to indicate the calculation of the accurateposition of the optical section 512 of the camera 510 mounted on theautomotive vehicle 508.

The conventional camera calibrating apparatus 500 further comprises afirst optical position information producing section 517 for producingfirst optical position information indicative of the position of theoptical section 512 in the first coordinate system 501, and a firstoptical position information storing section 518 for storing the firstoptical position information produced by the first optical positioninformation producing section 517. The first optical positioninformation producing section 517 is adapted to produce optical positioninformation indicative of the position of the optical section 512 in thefirst coordinate system 501 on the basis of the image information of therevising marker 505 obtained by the camera 510.

The conventional camera calibrating apparatus 500 further comprises asecond optical position information producing section 520 for producingsecond optical information indicative of the position of the opticalsection 512 in the second coordinate system 502, and a second opticalposition information storing section 530 for storing the second opticalposition information produced by the second optical position informationproducing section 520.

The second optical position information producing section 520 is adaptedto produce optical position information indicative of the position ofthe optical section 512 in the second coordinate system 502 from thesecond housing position information stored in the second housingposition information storing section 516 on the basis of the firsthousing position information stored in the first housing positioninformation storing section 515 and the first optical positioninformation stored in the first optical position information storingsection 518.

The conventional camera calibrating apparatus 500 thus constructed aspreviously mentioned is adapted to revise the camera in the cameraproducing plant by calculating the position of the optical section 512in the second coordinate system 502, while the image controllingapparatus is adapted to detect the position occupied by an objectlocated on the road through the image information obtained by the camera510 mounted on the automotive vehicle 508.

The conventional camera calibrating apparatus as previously mentioned,however, encounters such a problem that the second optical positioninformation stored in the second optical position information storingsection is not calibrated. This leads to the fact that the position ofthe object located on the road cannot be accurately detected by theimage controlling apparatus if the camera is mounted on an inaccurateposition of the automotive vehicle.

It is, therefore, an object of the present invention to provide a cameracalibrating apparatus which can calibrate optical parameters of theoptical section of the camera mounted on the automotive vehicle or thelike.

DISCLOSURE OF THE INVENTION

In accordance with the present invention, there is provided a cameracalibrating apparatus to be operative in combination with an imagingdevice which includes a housing unit and an optical section supported bythe housing unit to obtain image information thorough the opticalsection, and adapted to calibrate position information of the opticalsection, comprising: first housing position information storing meansfor storing first housing position information indicative of theposition of the housing unit in the first coordinate system; secondhousing position information storing means for storing second housingposition information indicative of the position of the housing unit inthe second coordinate system; first optical position informationproducing means for producing first optical position informationindicative of the position of the optical section in the firstcoordinate system on the basis of the image information obtained by theimaging device in the first coordinate system; first optical positioninformation storing means for storing the first optical positioninformation produced by the first optical position information producingmeans; second optical position information producing means for producingsecond optical position information indicative of the position of theoptical section in the second coordinate system from the second housingposition information stored by the second housing position informationstoring means on the basis of the first housing position informationstored by the first housing position information storing means and thefirst optical position information stored by the first optical positioninformation storing means; second optical position information storingmeans for storing the second optical position information produced bythe second optical position information producing means; and calibratingmeans for calibrating the second optical position information stored bythe second optical position information storing means on the basis ofthe image information obtained by the imaging device in the secondcoordinate system. The camera calibrating apparatus according to thepresent invention thus constructed as previously mentioned can calibrateparameters of the optical section mounted on an automotive vehicle orthe like.

In accordance with the present invention, there is provided a cameracalibrating apparatus to be operative in combination with an imagingdevice which includes a housing unit and an optical section supported bythe housing unit to obtain image information thorough the opticalsection, and adapted to calibrate optical position information about theposition of the optical section, comprising: optical positioninformation storing means for storing optical position informationindicative of the position of the optical section to a predeterminedcoordinate system; and calibrating means for calibrating the opticalposition information stored by the optical position information storingmeans on the basis of the image information obtained by the imagingdevice on the predetermined coordinate system. The camera calibratingapparatus according to the present invention thus constructed aspreviously mentioned can calibrate parameters of the optical sectionmounted on an automotive vehicle or the like.

In accordance with the present invention, there is provided a cameracalibrating apparatus to be operative in combination with an imagingdevice which includes a housing unit and an optical section supported bythe housing unit to obtain image information thorough the opticalsection, and adapted to calibrate optical position information about theposition of the optical section, comprising: first housing positioninformation storing means for storing first housing position informationindicative of the position of the housing unit in the first coordinatesystem in which a revising marker is located; second housing positioninformation storing means for storing second housing positioninformation indicative of the position of the housing unit in the secondcoordinate system in which a calibrating marker is located; firstoptical position information producing means for producing first opticalposition information indicative of the position of the optical sectionin the first coordinate system on the basis of the image information ofthe revising marker obtained by the imaging device; first opticalposition information storing means for storing the first opticalposition information produced by the first optical position informationproducing means; second optical position information producing means forproducing second optical position information indicative of the positionof the optical section in the second coordinate system from the secondhousing position information stored by the second housing positioninformation storing means on the basis of the first housing positioninformation stored by the first housing position information storingmeans and the first optical position information stored by the firstoptical position information storing means; estimated locationinformation producing means for producing estimated location informationindicative of the position of the calibrating marker to an imagecoordinate system of the imaging device on the basis of the secondoptical position information produced by the second optical positioninformation producing means; second optical position information storingmeans for storing the second optical position information produced bythe second optical position information producing means; estimatedlocation information storing means for storing the estimated locationinformation produced by the estimated location information estimatingmeans; and calibrating means for calibrating the second optical positioninformation stored by the second optical position information storingmeans on the basis of the image information of the calibrating markerobtained by the imaging device and the estimated location informationstored by the estimated location information storing means. The cameracalibrating apparatus according to the present invention thusconstructed as previously mentioned can calibrate the second opticalposition information by using a simple calibrating marker.

In the camera calibrating apparatus according to the present invention,the calibrating means includes: image location information extractingmeans for extracting an image location information indicative of animage location of the calibrating marker to the image coordinate systemof the imaging device on the basis of the image information of thecalibrating marker obtained by the imaging device; calibration valuecalculating means for calculating a calibration value of the secondoptical position information stored by the second optical positioninformation storing means on the basis of the image location informationextracted by the image location information extracting means and theestimated location information stored by the estimated locationinformation storing means; and optical position information calibratingmeans for calibrating the second optical position information stored bythe second optical position information storing means on the basis ofthe calibration value calculated by the calibration value calculatingmeans. The camera calibrating apparatus according to the presentinvention thus constructed as previously mentioned can calculatecalibration value of the second optical position information by using asimple calibrating marker.

In the camera calibrating apparatus according to the present invention,the calibrating means is adapted to calibrate a deviation of arotational component of the second optical position information. Thecamera calibrating apparatus according to the present invention thusconstructed as previously mentioned can calibrate the second opticalposition information by using a simple calibrating marker without takinginto account parallel shift component of the second optical positioninformation.

In the camera calibrating apparatus according to the present invention,the image location information extracting means includes: imagedisplaying means for displaying an image of the calibrating markerobtained by the imaging device, and image location specifying means forspecifying the image location of the calibrating marker on the image ofthe calibrating marker displayed by the image displaying means and toensure that the image location information is extracted. The cameracalibrating apparatus according to the present invention thusconstructed as previously mentioned can specify the image location ofthe calibrating marker to ensure that the image location information ofthe calibrating marker is extracted.

In the camera calibrating apparatus according to the present invention,the image location information extracting means includes: estimated areainformation storing means for storing estimated area informationindicative of the calibrating marker to the image coordinate system ofthe imaging device; and image location searching means for searching theimage location of the calibrating marker from the image information ofthe calibrating marker obtained by the imaging device on the basis ofthe estimated area information stored by the estimated area informationstoring means and the estimated location information stored by theestimated location information storing means to ensure that the imagelocation information is extracted. The camera calibrating apparatusaccording to the present invention thus constructed as previouslymentioned can search the image location of the calibrating marker toensure that the image location information of the calibrating marker iseasily extracted.

In accordance with the present invention, there is provided a cameracalibrating apparatus to be operative in combination with an imagingdevice which includes a housing unit and an optical section supported bythe housing unit to obtain image information thorough the opticalsection, and adapted to calibrate optical position information about theposition of the optical section, comprising: first housing positioninformation storing means for storing first housing position informationindicative of the position of the housing unit in the first coordinatesystem in which a revising marker is located; second housing positioninformation storing means for storing second housing positioninformation indicative of the position of the housing unit in the secondcoordinate system; first optical position information producing meansfor producing first optical position information indicative of theposition of the optical section in the first coordinate system on thebasis of the image information obtained by the imaging device andindicative of the revising marker; first optical position informationstoring means for storing the first optical position informationproduced by the first optical position information producing means;second optical position information producing means for producing secondoptical position information indicative of the position of the opticalsection in the second coordinate system from the second housing positioninformation stored by the second housing position information storingmeans on the basis of the first housing position information stored bythe first housing position information storing means and the firstoptical position information stored by the first optical positioninformation storing means; second optical position information storingmeans for storing the second optical position information produced bythe second optical position information producing means; and calibratingmeans for calibrating the second optical position information stored bythe second optical position information storing means on the basis of amotion vector of the image information obtained by the imaging device inthe second coordinate system. The camera calibrating apparatus accordingto the present invention thus constructed as previously mentioned cancalibrate the second optical position information by using the motionvector.

In the camera calibrating apparatus according to the present invention,the calibrating means includes: plane-projected image producing meansfor producing a plane-projected image from the image informationobtained by the imaging device in the second coordinate system;plane-projected image dividing means for dividing the plane-projectedimage produced by the plane-projected image producing means into aplurality of image segments; motion vector extracting means forextracting the motion vector from the image segments divided by theplane-projected image dividing means; calibration value calculatingmeans for calculating a calibration value of the second optical positioninformation stored by the second optical position information storingmeans on the basis of the motion vector extracted by the motion vectorextracting means; and optical position information calibrating means forcalibrating the second optical position information stored by the secondoptical position information storing means on the basis of thecalibration value calculated by the calibration value calculating means.The camera calibrating apparatus according to the present invention thusconstructed as previously mentioned can easily extract the motion vectorfrom the image segments.

In the camera calibrating apparatus according to the present invention,a dividing marker located in the second coordinate system in apredetermined relationship with the position of the housing unitrepresented by the second position information stored by the secondposition information storing means. The plane-projected image dividingmeans is adapted to divide the plane-projected image produced by theplane-projected image producing means into a plurality of image segmentson the basis of the image information of the dividing marker obtained bythe imaging device. The camera calibrating apparatus according to thepresent invention thus constructed as previously mentioned canaccurately divide the plane-projected image by using the dividingmarker.

In accordance with the present invention, there is provided a cameracalibrating apparatus to be operative in combination with an imagingdevice which includes a housing unit and an optical section supported bythe housing unit to obtain image information thorough the opticalsection, and adapted to calibrate position information of the opticalsection, comprising: first housing position information storing meansfor storing first housing position information indicative of theposition of the housing unit in the first coordinate system in which arevising marker is located; second housing position information storingmeans for storing second housing position information indicative of theposition of the housing unit in the second coordinate system in which anautomotive vehicle is located; first optical position informationproducing means for producing first optical position informationindicative of the position of the optical section in the firstcoordinate system on the basis of the image information of the revisingmarker obtained by the imaging device; first optical positioninformation storing means for storing the first optical positioninformation produced by the first optical position information producingmeans; second optical position information producing means for producingsecond optical position information indicative of the position of theoptical section in the second coordinate system from the second housingposition information stored by the second housing position informationstoring means on the basis of the first housing position informationstored by the first housing position information storing means and thefirst optical position information stored by the first optical positioninformation storing means; estimated location information producingmeans for producing estimated location information indicative of theposition of the automotive vehicle to an image coordinate system of theimaging device on the basis of the second optical position informationproduced by the second optical position information producing means;second optical position information storing means for storing the secondoptical position information produced by the second optical positioninformation producing means; estimated location information storingmeans for storing the estimated location information produced by theestimated location information estimating means; and calibrating meansfor calibrating the second optical position information stored by thesecond optical position information storing means on the basis of theimage information of the automotive vehicle obtained by the imagingdevice and the estimated location information stored by the estimatedlocation information storing means. The camera calibrating apparatusaccording to the present invention thus constructed as previouslymentioned can calibrate the second optical position information by usinga portion of the automotive vehicle.

In the camera calibrating apparatus according to the present invention,the calibrating means includes: image location information extractingmeans for extracting an image location information indicative of animage location of the automotive vehicle in the image coordinate systemof the imaging device on the basis of the image information of theautomotive vehicle obtained by the imaging device; calibration valuecalculating means for calculating a calibration value of the secondoptical position information stored by the second optical positioninformation storing means on the basis of the image location informationextracted by the image location information extracting means and theestimated location information stored by the estimated locationinformation storing means; and optical position information calibratingmeans for calibrating the second optical position information stored bythe second optical position information storing means on the basis ofthe calibration value calculated by the calibration value calculatingmeans. The camera calibrating apparatus according to the presentinvention thus constructed as previously mentioned can calculate thecalibration value of the second optical position information by using aportion of the automotive vehicle.

In the camera calibrating apparatus according to the present invention,the calibration value calculating means includes: superimposing meansfor superimposing a profile line of the automotive vehicle representedby the image location information on a profile line of the automotivevehicle represented by the estimated location information; extractingmeans for extracting a plurality of points from the overlapped profilelines of the automotive vehicle superimposed by the superimposing means,and calculating means for calculating a calibration value of the secondoptical position information by comparing the points of the imagelocation information with the points of the estimated locationinformation. The camera calibrating apparatus according to the presentinvention thus constructed as previously mentioned can extract thepoints from the profile line of the automotive vehicle to ensure thatthe second optical position information is extracted.

In the camera calibrating apparatus according to the present invention,the imaging device is mounted on an automotive vehicle. The cameracalibrating apparatus according to the present invention thusconstructed as previously mentioned can calibrate parameters of theoptical section to accurately detect the position of an object locatedon the road even if the housing unit is mounted on the inaccurateposition of the automotive vehicle.

In accordance with the present invention, there is provided an imagingsystem comprises a camera calibrating apparatus. The imaging systemaccording to the present invention thus constructed as previouslymentioned can calibrate parameters of the optical section mounted on anautomotive vehicle or the like.

In accordance with the present invention, there is provided an imagingcontrol system comprises a camera calibrating apparatus. The imagingcontrol system according to the present invention thus constructed aspreviously mentioned can calibrate parameters of the optical sectionmounted on an automotive vehicle or the like.

In accordance with the present invention, there is provided a cameracalibrating method of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, comprising: a first housing positioninformation storing step of storing first housing position informationindicative of the position of the housing unit in the first coordinatesystem; a second housing position information storing step of storingsecond housing position information indicative of the position of thehousing unit in the second coordinate system; a first optical positioninformation producing step of producing first optical positioninformation indicative of the position of the optical section in thefirst coordinate system on the basis of the image information obtainedby the imaging device in the first coordinate system; a first opticalposition information storing step of storing the first optical positioninformation produced in the first optical position information producingstep; a second optical position information producing step of producingsecond optical position information indicative of the position of theoptical section in the second coordinate system from the second housingposition information stored in the second housing position informationstoring step on the basis of the first housing position informationstored in the first housing position information storing step and thefirst optical position information stored in the first optical positioninformation storing step; a second optical position information storingstep of storing the second optical position information produced in thesecond optical position information producing step; and a calibratingstep of calibrating the second optical position information stored inthe second optical position information storing step on the basis of theimage information obtained by the imaging device in the secondcoordinate system. The camera calibrating method according to thepresent invention thus constructed as previously mentioned can calibrateparameters of the optical section mounted on an automotive vehicle orthe like to accurately detect the position of an object located on theroad.

In accordance with the present invention, there is provided a cameracalibrating method of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, comprising: an optical position informationstoring step of storing optical position information indicative of theposition of the optical section to a predetermined coordinate system;and a calibrating step of calibrating the optical position informationstored in the optical position information storing step on the basis ofthe image information obtained by the imaging device on thepredetermined coordinate system. The camera calibrating method accordingto the present invention thus constructed as previously mentioned cancalibrate parameters of the optical section mounted on an automotivevehicle or the like to accurately detect the position of an objectlocated on the road.

In accordance with the present invention, there is provided a cameracalibrating method of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, comprising: a first housing positioninformation storing step of storing first housing position informationindicative of the position of the housing unit in the first coordinatesystem in which a revising marker is located; a second housing positioninformation storing step of storing second housing position informationindicative of the position of the housing unit in the second coordinatesystem in which a calibrating marker is located; a first opticalposition information producing step of producing first optical positioninformation indicative of the position of the optical section in thefirst coordinate system on the basis of the image information of therevising marker obtained by the imaging device; a first optical positioninformation storing step of storing the first optical positioninformation produced in the first optical position information producingstep; a second optical position information producing step of producingsecond optical position information indicative of the position of theoptical section in the second coordinate system from the second housingposition information stored in the second housing position informationstoring step on the basis of the first housing position informationstored in the first housing position information storing step and thefirst optical position information stored in the first optical positioninformation storing step; an estimated location information producingstep of producing estimated location information indicative of theposition of the calibrating marker to an image coordinate system of theimaging device on the basis of the second optical position informationproduced in the second optical position information producing step; asecond optical position information storing step of storing the secondoptical position information produced in the second optical positioninformation producing step; an estimated location information storingstep of storing the estimated location information produced in theestimated location information estimating step; and a calibrating stepof calibrating the second optical position information stored in thesecond optical position information storing step on the basis of theimage information of the calibrating marker obtained by the imagingdevice and the estimated location information stored in the estimatedlocation information storing step. The camera calibrating methodaccording to the present invention thus constructed as previouslymentioned can calibrate the second optical position information by usingthe simple calibrating marker.

In accordance with the present invention, there is provided a cameracalibrating method of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, comprising: a first housing positioninformation storing step of storing first housing position informationindicative of the position of the housing unit in the first coordinatesystem in which a revising marker is located; a second housing positioninformation storing step of storing second housing position informationindicative of the position of the housing unit in the second coordinatesystem; a first optical position information producing step of producingfirst optical position information indicative of the position of theoptical section in the first coordinate system on the basis of the imageinformation obtained by the imaging device and indicative of therevising marker; a first optical position information storing step ofstoring the first optical position information produced in the firstoptical position information producing step; a second optical positioninformation producing step of producing second optical positioninformation indicative of the position of the optical section in thesecond coordinate system from the second housing position informationstored in the second housing position information storing step on thebasis of the first housing position information stored in the firsthousing position information storing step and the first optical positioninformation stored in the first optical position information storingstep; a second optical position information storing step of storing thesecond optical position information produced in the second opticalposition information producing step; and a calibrating step ofcalibrating the second optical position information stored in the secondoptical position information storing step on the basis of a motionvector of the image information obtained by the imaging device in thesecond coordinate system. The camera calibrating method according to thepresent invention thus constructed as previously mentioned can calibratethe second optical position information by using the motion vector.

In accordance with the present invention, there is provided a cameracalibrating method of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, comprising: a first housing positioninformation storing step of storing first housing position informationindicative of the position of the housing unit in the first coordinatesystem in which a revising marker is located; a second housing positioninformation storing step of storing second housing position informationindicative of the position of the housing unit in the second coordinatesystem in which an automotive vehicle is located; a first opticalposition information producing step of producing first optical positioninformation indicative of the position of the optical section in thefirst coordinate system on the basis of the image information of therevising marker obtained by the imaging device; a first optical positioninformation storing step of storing the first optical positioninformation produced in the first optical position information producingstep; a second optical position information producing step of producingsecond optical position information indicative of the position of theoptical section in the second coordinate system from the second housingposition information stored in the second housing position informationstoring step on the basis of the first housing position informationstored in the first housing position information storing step and thefirst optical position information stored in the first optical positioninformation storing step; an estimated location information producingstep of producing estimated location information indicative of theposition of the automotive vehicle to an image coordinate system of theimaging device on the basis of the second optical position informationproduced in the second optical position information producing step; asecond optical position information storing step of storing the secondoptical position information produced in the second optical positioninformation producing step; an estimated location information storingstep of storing the estimated location information produced in theestimated location information estimating step; and a calibrating stepof calibrating the second optical position information stored in thesecond optical position information storing step on the basis of theimage information of the automotive vehicle obtained by the imagingdevice and the estimated location information stored in the estimatedlocation information storing step. The camera calibrating methodaccording to the present invention thus constructed as previouslymentioned can calibrate the second optical position information by usinga portion of the automotive vehicle.

In the camera calibrating apparatus according to the present invention,the imaging device is mounted on an automotive vehicle. The cameracalibrating method according to the present invention thus constructedas previously mentioned can calibrate parameters of the optical sectionmounted on an automotive vehicle or the like to accurately detect theposition of an object located on the road even if the housing unit ismounted on the inaccurate position of the automotive vehicle.

In accordance with the present invention, there is provided a cameracalibrating program of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, and allowing a computer to execute: a firsthousing position information storing step of storing first housingposition information indicative of the position of the housing unit inthe first coordinate system; a second housing position informationstoring step of storing second housing position information indicativeof the position of the housing unit in the second coordinate system; afirst optical position information producing step of producing firstoptical position information indicative of the position of the opticalsection in the first coordinate system on the basis of the imageinformation obtained by the imaging device in the first coordinatesystem; a first optical position information storing step of storing thefirst optical position information produced in the first opticalposition information producing step; a second optical positioninformation producing step of producing second optical positioninformation indicative of the position of the optical section in thesecond coordinate system from the second housing position informationstored in the second housing position information storing step on thebasis of the first housing position information stored in the firsthousing position information storing step and the first optical positioninformation stored in the first optical position information storingstep; a second optical position information storing step of storing thesecond optical position information produced in the second opticalposition information producing step; and a calibrating step ofcalibrating the second optical position information stored in the secondoptical position information storing step on the basis of the imageinformation obtained by the imaging device in the second coordinatesystem. The camera calibrating program according to the presentinvention thus constructed as previously mentioned can calibrateparameters of the optical section mounted on an automotive vehicle orthe like to accurately detect the position of an object located on theroad.

In accordance with the present invention, there is provided a cameracalibrating program of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, and allowing a computer to execute: anoptical position information storing step of storing optical positioninformation indicative of the position of the optical section to apredetermined coordinate system; and a calibrating step of calibratingthe optical position information stored in the optical positioninformation storing step on the basis of the image information obtainedby the imaging device on the predetermined coordinate system. The cameracalibrating program according to the present invention thus constructedas previously mentioned can calibrate parameters of the optical sectionmounted on an automotive vehicle or the like to accurately detect theposition of an object located on the road.

In accordance with the present invention, there is provided a cameracalibrating program of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, and allowing a computer to execute: a firsthousing position information storing step of storing first housingposition information indicative of the position of the housing unit inthe first coordinate system in which a revising marker is located; asecond housing position information storing step of storing secondhousing position information indicative of the position of the housingunit in the second coordinate system in which a calibrating marker islocated; a first optical position information producing step ofproducing first optical position information indicative of the positionof the optical section in the first coordinate system on the basis ofthe image information of the revising marker obtained by the imagingdevice; a first optical position information storing step of storing thefirst optical position information produced in the first opticalposition information producing step; a second optical positioninformation producing step of producing second optical positioninformation indicative of the position of the optical section in thesecond coordinate system from the second housing position informationstored in the second housing position information storing step on thebasis of the first housing position information stored in the firsthousing position information storing step and the first optical positioninformation stored in the first optical position information storingstep; an estimated location information producing step of producingestimated location information indicative of the position of thecalibrating marker to an image coordinate system of the imaging deviceon the basis of the second optical position information produced in thesecond optical position information producing step; a second opticalposition information storing step of storing the second optical positioninformation produced in the second optical position informationproducing step; an estimated location information storing step ofstoring the estimated location information produced in the estimatedlocation information estimating step; and a calibrating step ofcalibrating the second optical position information stored in the secondoptical position information storing step on the basis of the imageinformation of the calibrating marker obtained by the imaging device andthe estimated location information stored in the estimated locationinformation storing step. The camera calibrating program according tothe present invention thus constructed as previously mentioned cancalibrate the second optical position information by using the simplecalibrating marker.

In accordance with the present invention, there is provided a cameracalibrating program of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, and allowing a computer to execute: a firsthousing position information storing step of storing first housingposition information indicative of the position of the housing unit inthe first coordinate system in which a revising marker is located; asecond housing position information storing step of storing secondhousing position information indicative of the position of the housingunit in the second coordinate system; a first optical positioninformation producing step of producing first optical positioninformation indicative of the position of the optical section in thefirst coordinate system on the basis of the image information obtainedby the imaging device and indicative of the revising marker; a firstoptical position information storing step of storing the first opticalposition information produced in the first optical position informationproducing step; a second optical position information producing step ofproducing second optical position information indicative of the positionof the optical section in the second coordinate system from the secondhousing position information stored in the second housing positioninformation storing step on the basis of the first housing positioninformation stored in the first housing position information storingstep and the first optical position information stored in the firstoptical position information storing step; a second optical positioninformation storing step of storing the second optical positioninformation produced in the second optical position informationproducing step; and a calibrating step of calibrating the second opticalposition information stored in the second optical position informationstoring step on the basis of a motion vector of the image informationobtained by the imaging device in the second coordinate system. Thecamera calibrating program according to the present invention thusconstructed as previously mentioned can calibrate the second opticalposition information by using the motion vector.

In accordance with the present invention, there is provided a cameracalibrating program of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, and allowing a computer to execute: a firsthousing position information storing step of storing first housingposition information indicative of the position of the housing unit inthe first coordinate system in which a revising marker is located; asecond housing position information storing step of storing secondhousing position information indicative of the position of the housingunit in the second coordinate system in which an automotive vehicle islocated; a first optical position information producing step ofproducing first optical position information indicative of the positionof the optical section in the first coordinate system on the basis ofthe image information of the revising marker obtained by the imagingdevice; a first optical position information storing step of storing thefirst optical position information produced in the first opticalposition information producing step; a second optical positioninformation producing step of producing second optical positioninformation indicative of the position of the optical section in thesecond coordinate system from the second housing position informationstored in the second housing position information storing step on thebasis of the first housing position information stored in the firsthousing position information storing step and the first optical positioninformation stored in the first optical position information storingstep; an estimated location information producing step of producingestimated location information indicative of the position of theautomotive vehicle to an image coordinate system of the imaging deviceon the basis of the second optical position information produced in thesecond optical position information producing step; a second opticalposition information storing step of storing the second optical positioninformation produced in the second optical position informationproducing step; an estimated location information storing step ofstoring the estimated location information produced in the estimatedlocation information estimating step; and a calibrating step ofcalibrating the second optical position information stored in the secondoptical position information storing step on the optical section basisof the image information of the automotive vehicle obtained by theimaging device and the estimated location information stored in theestimated location information storing step. The camera calibratingprogram according to the present invention thus constructed aspreviously mentioned can calibrate the second optical positioninformation by using a portion of the automotive vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

The features and advantages of a camera calibrating apparatus accordingto the present invention will be more clearly understood from thefollowing description taken in conjunction with the accompanyingdrawings:

FIG. 1 is a block diagram showing a camera calibrating apparatusaccording to the first embodiment of the present invention and a cameraexemplified as an imaging device;

FIG. 2 is a perspective view showing the first coordinate system inwhich the camera shown in FIG. 1 is located;

FIG. 3 is a perspective view showing the second coordinate system inwhich the camera shown in FIG. 1 is located;

FIG. 4 is a perspective view showing a coordinate system of the camerashown in FIG. 1;

FIG. 5 is a plan view showing an image coordinate system of the camerashown in FIG. 1;

FIG. 6 is a perspective view showing a parallel translation of thecamera shown in FIG. 1;

FIG. 7 is a perspective view showing a rotational motion of the camerashown in FIG. 1;

FIG. 8 is a side view showing a parallel translation of the camera shownin FIG. 1;

FIG. 9 is a side view showing a rotational motion of the camera shown inFIG. 1;

FIG. 10 is a block diagram showing an image location informationextracting section of the camera calibrating apparatus shown in FIG. 1;

FIG. 11 is a block diagram showing a computer and ECU to ensure thecamera calibrating apparatus shown in FIG. 1;

FIG. 12 is a block diagram showing a camera unit of the cameracalibrating apparatus shown in FIG. 1;

FIG. 13 is a flow chart showing a calibrating operation of the cameracalibrating apparatus shown in FIG. 1;

FIG. 14 is a block diagram showing a camera calibrating apparatusaccording to the second embodiment of the present invention and a cameraexemplified as an imaging device;

FIG. 15 a plan view showing an image coordinate system of the camerashown in FIG. 14;

FIG. 16 is a flow chart showing a calibrating operation of the cameracalibrating apparatus shown in FIG. 14;

FIG. 17 is a block diagram showing the camera calibrating apparatusaccording to the third embodiment of the present invention and a cameraexemplified as an imaging device;

FIG. 18 is a perspective view showing the first coordinate system inwhich the camera shown in FIG. 17 is located;

FIG. 19 is a perspective view showing the second coordinate system inwhich the camera shown in FIG. 17 is located;

FIG. 20 is a perspective view showing a coordinate system of the camerashown in FIG. 17;

FIG. 21 is a side view showing an imaginary camera of the cameracalibrating apparatus shown in FIG. 17;

FIG. 22 is a schematic view showing an operation of the calibratingsection of the camera calibrating apparatus shown in FIG. 17;

FIG. 23 is a schematic view showing an operation of the calibratingsection of the camera calibrating apparatus shown in FIG. 17;

FIG. 24 is a schematic view showing an operation of the calibratingsection of the camera calibrating apparatus shown in FIG. 17;

FIG. 25 is a schematic view showing an operation of the calibratingsection of the camera calibrating apparatus shown in FIG. 17;

FIG. 26 is a block diagram showing a computer and ECU to ensure thecamera calibrating apparatus shown in FIG. 17;

FIG. 27 is a block diagram showing a camera unit of the cameracalibrating apparatus shown in FIG. 17;

FIG. 28 is a flow chart showing a calibrating operation of the cameracalibrating apparatus shown in FIG. 17;

FIG. 29 is a block diagram showing a camera calibrating apparatusaccording to the fourth embodiment of the present invention and a cameraexemplified as an image device;

FIG. 30 is a perspective view showing the first coordinate system inwhich the camera shown in FIG. 29 is located;

FIG. 31 is a perspective view showing the second coordinate system inwhich the camera shown in FIG. 29 is located;

FIG. 32 is a perspective view showing a coordinate system of the camerashown in FIG. 29;

FIG. 33 is a plan view showing an image coordinate system of the camerashown in FIG. 29;

FIG. 34 is a schematic view showing an operation of the calibratingsection of the camera calibrating apparatus shown in FIG. 29;

FIG. 35 is a schematic view showing an operation of the calibratingsection of the camera calibrating apparatus shown in FIG. 29;

FIG. 36 is a block diagram showing a computer and ECU to ensure thecamera calibrating apparatus shown in FIG. 29;

FIG. 37 is a block diagram showing a camera unit of the cameracalibrating apparatus shown in FIG. 29;

FIG. 38 is a flow chart showing a calibrating operation of the cameracalibrating apparatus shown in FIG. 29;

FIG. 39 is a block diagram of a conventional camera calibratingapparatus and a camera exemplified as an imaging device;

FIG. 40 is a perspective view showing the first coordinate system inwhich the camera shown in FIG. 39 is located; and

FIG. 41 is a perspective view showing the second coordinate system inwhich the camera shown in FIG. 39 is located.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The embodiments of the camera calibrating apparatus according to thepresent invention will now be described in detail in accordance withaccompanying drawings.

First Embodiment

Referring now to the drawings, in particular to FIGS. 1 to 13, there isshown a first embodiment of a camera calibrating apparatus according tothe present invention.

The construction of the first embodiment of the camera calibratingapparatus according to the present invention will now be describedhereinafter. As shown in FIGS. 1 to 3, the camera calibrating apparatus100 is electrically connected to a camera 110 exemplified as an imagingdevice. The camera 110 includes a housing unit 111 and an opticalsection 112 supported by the housing unit 111 to obtain imageinformation through the optical section 112.

The camera calibrating apparatus 100 comprises a first housing positioninformation storing section 115 for storing first housing positioninformation indicative of the position of the housing unit 111 in thefirst coordinate system 101, a second housing position informationstoring section 116 for storing second housing position informationindicative of the position of the housing unit 111 in the secondcoordinate system 102, a revising marker position information storingsection 125 for storing revising marker position information indicativeof the position of a revising marker 105 in the first coordinate system101, and a calibrating marker position information storing section 126for storing calibrating marker position information indicative of theposition of a calibrating marker 106 in the second coordinate system102.

The first coordinate system 101 is defined in a first working space suchas a camera production plant to have X₁-axis, Y₁-axis, and Z₁-axis,while the revising marker 105 laid out in the first working space causesthe camera calibrating apparatus 100 to revise the camera 110 located inthe first working space. The revising marker 105 is constituted by aplurality of marking objects which are respectively located inpredetermined positions on the first coordinate system 101, and whichare within a viewing field of the camera 110 located in the firstworking space.

The second coordinate system 102 is defined in a second working spacesuch as an automotive vehicle production plant to have X₂-axis, Y₂-axis,and Z₂-axis, while X₂-axis and Y₂-axis collectively define X₂-Y₂ planewhich is represented by a road surface 102 a on which an automotivevehicle 108 is located. The calibrating marker 106 is constituted by twoor more marking objects which are respectively located at predeterminedpositions of the road surface 102 a in the second coordinate system 102.

The camera calibrating apparatus 100 is adapted to revise the camera 110in the first working space. The camera 110 is located at a predeterminedposition on the first coordinate system 101, while the first housingposition information storing section 115 is adapted to store firsthousing position information indicative of the position of the housingunit 111. Here, the term “the calibration of the camera 110” is intendedto indicate the calculation of the accurate position of the opticalsection 112 of the camera 110 mounted on the automotive vehicle 108.

The camera 110 revised by the camera calibrating apparatus 100 ismounted on the automotive vehicle 108 to be located at a predeterminedposition on the second coordinate system 102 in the second workingspace, while the second housing position information storing section 116is adapted to store second housing position information indicative ofthe position of the housing unit 111. Here, the second housing positioninformation indicates an accurate position in which the housing unit 111of the camera 110 is located when the camera 110 is mounted on theautomotive vehicle 108.

The camera calibrating apparatus 100 further comprises a first opticalposition information producing section 117 for producing first opticalposition information indicative of the position of the optical section112 in the first coordinate system 101, and a first optical positioninformation storing section 118 for storing the first optical positioninformation produced by the first optical position information producingsection 117.

The first optical position information producing section 117 is adaptedto calculate the position of the optical section 112 in the firstcoordinate system 101 from the calibrating marker position informationstored in the calibrating marker position information storing section125 on the basis of the image information of the calibrating marker 105obtained by the camera 110. Here, the term “the position of the opticalsection 112” is intended to indicate an optical center of the opticalsection 112 and the position of an optical axis, and other opticalcoordinate parameters. The calculation of the position of the opticalsection 112 in the first coordinate system 101 is performed on the basisof a method disclosed in the document 1 (R. Tsai, A versatile cameracalibration technique for high-accuracy 3D machine vision metrologyusing off-the-shelf TV cameras and lenses, IEEE Journal of Robotics andAutomation, RA-3 (4): 323-344, 1987).

The camera calibrating apparatus 100 further comprises a second opticalposition information producing section 120 for producing second opticalinformation indicative of the position of the optical section 112 in thesecond coordinate system 102, and a second optical position informationstoring section 130 for storing the second optical position informationproduced by the second optical position information producing section120.

The second optical position information producing section 120 is adaptedto produce second optical position information indicative of theposition of the optical section 112 in the second coordinate system 102from the second housing position information stored in the secondhousing position information storing section 116 on the basis of thefirst housing position information stored by the first housing positioninformation storing section 115 and the first optical positioninformation stored in the first optical position information storingsection 118.

The second optical position information producing section 120 is adaptedto calculate the position of the optical section 112 in the secondcoordinate system 102 on the basis of a following method.

The method is of comparing the position of the housing unit 111 to thefirst coordinate system 101 with the position of the optical section 112in the first coordination system 101 to obtain positional informationbetween the housing unit 111 and the optical section 112, and tocalculate information indicative of the position of the optical section112 in the second coordinate system 102 from the position of the housingunit 111 in the second coordinate system 102 on the basis of thepositional information between the housing unit 111 and the opticalsection 112. Consequently, the second optical position informationindicates the position of the optical section 112 under the conditionthat the housing unit 111 is located at an accurate position of theautomotive vehicle 108.

As shown in FIG. 4, the camera 110 mounted on the automotive vehicle 108in the second working space defines a camera coordinate system 113having x-axis, y-axis, and z-axis on the basis of the second opticalposition information. The origin point of the camera coordinate system113 is in register with the optical center of the optical section 112.The x-axis of the camera coordinate system 113 extends in a horizontaldirection of the camera 110. The y-axis of the camera coordinate system113 extends in a vertical direction of the camera 110. The z-axis of thecamera coordinate system 113 is in axial alignment with the optical axisof the optical section 112.

The image coordinate system 114 is defined on a plane spaced apart fromthe origin point of the camera coordinate system 113 with a focal length“f” in the direction of the z-axis of the camera coordinate system 113.The image coordination system 114 has p-axis and q-axis. The camera 110is adapted to obtain, as image information, an optical image focused onthe image coordinate system 114 through the optical section 112.

The camera calibrating apparatus 100 further comprises an estimatedposition information producing section 140 for producing estimatedposition information of the calibrating marker 106 to the imagecoordinate system 114 of the camera 110, and an estimated positioninformation storing section 150 for storing the estimated positioninformation produced by the estimated position information producingsection 140.

The estimated position information producing section 140 is adapted toproduce estimated position information of the calibrating marker 106 tothe image coordinate system 114 of the camera 110 from the positioninformation of the calibrating marker 106 stored in the calibratingmarker position information storing section 126 on the basis of thesecond optical position information produced by the second opticalposition information producing section 120. The calculation of theestimated location of the calibrating marker 106 to the image coordinatesystem 114 of the camera 110 is performed on the basis of the abovementioned document 1.

As shown in FIG. 5, the calibrating marker 106 located in the secondcoordinate system 102 is focused on the image location Pn′ (n=1, 2, 3,4, 5, 6) of the image coordinate system 114 through the optical section112. Here, the image location Pn′ is in register with the estimatedimage location Pn (n=1, 2, 3, 4, 5, 6) calculated by the estimated imagelocation calculating section 140 under the condition that the housingunit 111 is located at an accurate position of the automotive vehicle108, more specifically, the housing unit 111 is located at the positionrepresented by the second housing position information with no deviationto the accurate position. However, the housing unit 111, in general, islocated at an inaccurate position with deviation to the accurateposition. This leads to the fact that the second optical positioninformation is produced with deviation to the accurate position. As aresult, the estimated location Pn is spaced apart from the imagelocation Pn′ of the image coordinate system 114.

The second optical position information includes six parametersindicative of a parallel displacement and a rotational motion of thecamera coordinate system 113 to the second coordinate system 102. Thesix parameters are constituted by the parallel displacement componentsof the X₂-axis, the Y₂-axis, and the Z₂-axis shown in FIG. 6 and therotational motion of the x-axis, the y-axis, and the z-axis shown inFIG. 7. The deviation of the second optical position informationincludes components of the parallel displacement and rotational motion.The housing unit 111 is, in general, located on the automotive vehiclewith the parallel displacement which is within the range of a fewcentimeters, and the rotational motion which is within the range of afew degrees.

The following description will be directed to the case that the imageinformation of the road surface 102 a obtained by the camera 110 isdisplayed in conjunction with an additional line when the automotivevehicle 108 is operated by an operator. Here, the housing unit 111 islocated at the position of the automotive vehicle 108 which is spacedapart from the road surface 302 a, while the additional line is locatedat the position spaced apart from a rear portion of the automotivevehicle 108. The height of the housing unit 111 to the road surface 302a is equal to 1,000 millimeters, while the distance of the additionalline to the rear portion is equal to 3000 millimeters.

When the housing unit 111 is located at the position spaced apart from adesign position of the automotive vehicle 108 in the Y-axis direction,more specifically, the parallel shift displacement of the housing unit111 to the design position is equal to 50 millimeters as shown in FIG.8, the displacement of the additional line is equal to 50 millimeters inY₂-axis direction. In this case, the elongation value of the imagelocation Pn′ to the estimated location Pn on the image coordinate system114 shown in FIG. 5 can be negligible.

When, on the other hand, the housing unit 111 is mounted on theautomotive vehicle 108 with a rotational displacement of the housingunit 111 to the X₂-axis direction, more specifically, the rotationaldisplacement of the housing unit 111 to the X₂-axis direction is equalto one degree as shown in FIG. 9, the additional line is located with adisplacement of 184 millimeters in the Y₂-axis direction. In this case,the elongation value of the image location Pn′ to the estimated locationPn on the image coordinate system 114 shown in FIG. 5 cannot benegligible.

As will be seen from the above description, the second optical positioninformation may be calibrated in consideration with only the rotationaldisplacement of the optical section 112.

The camera calibrating apparatus 100 further comprises a calibratingsection 160 for calibrating the second optical position informationstored in the second optical position information storing section 130 toorder to calibrate the above mentioned deviation of the second opticalposition information.

The calibrating section 160 is adapted to calibrate the second opticalposition information stored in the second optical position informationstoring section 130 on the basis of the image information of thecalibrating marker 106 obtained by the camera 110 and the estimatedposition information stored in the estimated position informationstoring section 150.

The calibrating section 160 includes an image location informationextracting section 170 for extracting image location informationindicative of an image location of the calibrating marker 106 to theimage coordinate system 114 of the camera 110 on the basis of the imageinformation of the calibrating marker 106 obtained by the camera 110, acalibration value calculating section 180 for calculating a calibrationvalue of the second optical position information stored in the secondoptical position information storing section 130 on the basis of theimage location information extracted by the image location informationextracting section 170 and the estimated position information stored inthe estimated position information storing section 150, and an opticalposition information calibrating section 190 for calibrating the secondoptical position information stored in the second optical positioninformation storing section 130 on the basis of the calibration valuecalculated by the calibration value calculating section 180.

The image location information extracting section 170 includes an imagedisplaying section 171 for displaying an image of the calibrating marker106 obtained by the camera 110, and an image location specifying section172 for specifying the image location Pn′ of the calibrating marker 106in the image of the calibrating marker 106 displayed by the imagedisplaying means 171 in order to extract the image location informationfrom the image information.

As shown in FIG. 10, the image displaying section 171 is adapted todisplay both the image of the calibrating marker 106 obtained by thecamera 110 and a cursor 174 for specifying the image location Pn′ of thecalibrating marker 106.

The image location specifying section 172 includes an up-arrow key 175a, a down-arrow key 175 b, a left-arrow key 175 c, and a right-arrow key175 d, all of which are capable of allowing the cursor 174 displayed onthe image information displaying section 171 to be moved in therespective directions, an enter key 176 for deciding the position of thecursor 174 as the image location Pn′ of the calibrating marker 106, anda number displaying section 177 for displaying number “n” of the markingobject of the calibrating marker 106, and an increment key 178 a and adecrement key 178 b each for changing the number “n” displayed on thenumber displaying section 177.

The calibration value calculating section 180 is adapted to calculatethe calibration value of the second optical position information on thebasis of the following method.

The following direction will be directed to the case that the deviationof the rotational component of the second optical position informationis considered as an overall deviation of the second optical positioninformation.

$\begin{matrix}{J = {\sum\limits_{n = 1}^{N}\begin{bmatrix}{\left( {{p_{n}^{\prime}\left( {{R_{13}p_{n}} + {R_{23}q_{n}} + {R_{33}f}} \right)} - {f\left( {{R_{11}p} + {R_{21}q} + {R_{31}f}} \right)}} \right)^{2} +} \\\left( {{q_{n}^{\prime}\left( {{R_{13}p} + {R_{23}q_{n}} + {R_{33}f}} \right)} - {f\left( {{R_{12}p} + {R_{22}q} + {R_{32}f}} \right)}} \right)^{2}\end{bmatrix}}} & (1)\end{matrix}$

(p_(n)′, q_(n)′) and (p_(n), q_(n)) are linked by the following formula(2) on the basis of the formula (1).

$\begin{matrix}\begin{matrix}{p^{\prime} = {f\frac{{R_{11}p} + {R_{21}q} + {R_{31}f}}{{R_{13}p} + {R_{23}q} + {R_{33}f}}}} \\{q^{\prime} = {f\frac{{R_{12}p} + {R_{22}q} + {R_{32}f}}{{R_{13}p} + {R_{23}q} + {R_{33}f}}}}\end{matrix} & (2)\end{matrix}$

R₁₁ to R₃₃ and θ, Φ, φ are linked by the following formulas (3), (4),and (5).

$\begin{matrix}\begin{matrix}{{Rx} = \begin{bmatrix}1 & 0 & 0 \\0 & {\cos\;\theta} & {\sin\;\theta} \\0 & {{- \sin}\;\theta} & {\cos\;\theta}\end{bmatrix}} \\{{Ry} = \begin{bmatrix}{\cos\;\phi} & 0 & {{- \sin}\;\phi} \\0 & 1 & 0 \\{\sin\;\phi} & 0 & {\cos\;\phi}\end{bmatrix}} \\{{Rz} = \begin{bmatrix}{\cos\;\varphi} & {\sin\;\varphi} & 0 \\{{- \sin}\;\varphi} & {\cos\;\varphi} & 0 \\0 & 0 & 1\end{bmatrix}}\end{matrix} & (3) \\{R = {{Rx} \times {Ry} \times {Rz}}} & (4) \\{R = \begin{bmatrix}R_{11} & R_{12} & R_{13} \\R_{21} & R_{22} & R_{23} \\R_{31} & R_{32} & R_{33}\end{bmatrix}} & (5)\end{matrix}$

In this embodiment, the calibrating marker 106 is constituted by sixmarking objects, however, the calibrating marker 106 may be constitutedby two or more marking objects. On the other hand, the calibration valuecalculating section 180 can calculate θ, Φ, and φ if the calibratingmarker 106 is constituted by two marking objects. Additionally, thecalibration value calculating section 180 can calculate θ, Φ, and φ at arelatively high accuracy if the calibrating marker 106 is constituted bythree or more marking objects.

As shown in FIG. 11, the camera calibrating apparatus 100 thusconstructed as previously mentioned can be realized by comprising thecomputer 191 for calibrating the camera 110, and an electronic controlunit (hereinafter simply referred to as “ECU”) 192 constituted as animage controlling apparatus for controlling the camera 110.

The computer 191 is shown in FIG. 11( a) as including a centralprocessing unit (hereinafter simply referred to as “CPU”), a randomaccess memory (hereinafter simply referred to as “RAM”), a read onlymemory (hereinafter simply referred to as “ROM”), an input/outputinterface, and other elements. The computer 191 is electricallyconnected to the camera 110 at the first working space. In thisembodiment, the computer 191 constitutes each of the first housingposition information storing section 115, the second housing positioninformation storing section 116, the first optical position informationproducing section 117, the first optical position information storingsection 118, the second optical position information producing section120, the calibrating marker position information storing section 125,the calibrating marker position information storing section 126, thesecond optical position storing section 130, the estimated positioninformation producing section 140, and the estimated positioninformation storing section 150.

The ECU 192 is shown in FIG. 11( c) as including CPU, RAM, ROM, aninput/output interface, and other elements. The ECU192 is mounted on theautomotive vehicle 108 to be electrically connected to the camera 110 inthe second working space. In this embodiment, the ECU 192 constituteseach of the second optical position information storing section 130, theestimated position information storing section 150, and the calibratingsection 160.

As will be seen from FIG. 11( b), the camera 110 is transferred to thesecond working space from the first working space with a recordablemedium 193 such as for example compact disk read only memory(hereinafter simply referred to as “CD-ROM” and a magnetic disk. Therecordable medium 193 has stored therein each of the second opticalposition information and the estimated position information to ensurethat the each of the second optical position information and theestimated position information is effectively transferred to the ECU 192from the computer 191.

In this embodiment, the camera 110 and the recordable medium 193 aretransferred to the second working space from the first working space,however, the camera 110, the second optical position information storingsection 130, the estimated position information storing section 150, andthe calibrating section 160 may be collectively transferred as a cameraassembly 194 to the second working space from the first working space aswill be seen from FIG. 12.

The following description will be directed to the operation of the firstembodiment of the camera calibrating apparatus according to the presentinvention. The second optical position information is calibrated by thecamera calibrating apparatus 100 through the steps shown in the FIG. 13.

The camera 110 is firstly located at the predetermined position on thefirst coordinate system 101 in the first working space (in the stepS101). The first housing position information, the second housingposition information, the revising marker position information, and thecalibrating marker position information are then stored in the firsthousing position information storing section 115, the second housingposition information storing section 116, the revising marker positioninformation storing section 125, and the calibrating marker positioninformation storing section 126 (in the step S102), respectively. Here,the first housing position information, the second housing positioninformation, the revising marker position information, and thecalibrating marker position information are obtained on the basis ofpositions predetermined in the design process and measured by ameasuring instrument.

The image information of the revising marker 105 is then obtained by thecamera 110 (in the step S103). The first optical position information isthen produced by the first optical position information producingsection 117 from the revising marker position information stored in therevising marker position information storing section 125 on the basis ofthe image information of the revising marker 105 obtained by the camera110 (in the step S104). The first optical position information producedby the first optical position information producing section 117 is thenstored in the first optical position information storing section 118 (inthe step S105).

The second optical position information is then produced by the secondoptical position information producing section 120 from the secondhousing position information stored in the second housing positioninformation storing section 116 on the basis of the first housingposition information stored in the first housing position informationstoring section 115 and the first optical position information stored inthe first optical position information storing section 118 (in the stepS106). The second optical position information produced by the secondoptical position information producing section 120 is then stored in thesecond optical position information storing section 130 (in the stepS107).

The estimated position information indicative of the estimated locationof the calibrating marker 106 is then produced by the estimated positioninformation producing section 140 from the calibrating marker positioninformation stored in the calibrating marker position informationstoring section 126 on the basis of the second optical positioninformation produced by the second optical position informationproducing section 120 (in the step S108). The estimated positioninformation produced by the estimated position information producingsection 140 is stored in the estimated position information storingsection 150 (in the step S109).

The camera 110 and the recordable medium 193 are then transferred to thesecond working space from the first working space. The camera 110 isthen mounted to the automotive vehicle 108 to be located at thepredetermined position on the second coordinate system 102 (in the stepS10).

The image information of the calibrating marker 106 is then obtained bythe camera 110 (in the step S111). The image information of thecalibrating marker 106 obtained by the camera 110 is then displayed onthe image displaying section 171 (in the step S112) as shown in FIG. 10.The image location Pn′ of the calibrating marker is specified by theimage location specifying section 172 (in the step S113). Here, theincrement key 178 a and the decrement key 178 b are operated by anoperator, while the number “n” of the marking object of the calibratingmarker 106 displayed by the number displaying section 177 in response tothe increment key 178 a and the decrement key 178 b. The up-arrow key175 a, the down-arrow key 175 b, the left-arrow key 175 c, and theright-arrow key 175 d are operated by the operator, while the cursor 174displayed on the image displaying section 171 is moved in response toeach of the up-arrow key 175 a, the down-arrow key 175 b, the left-arrowkey 175 c, and the right-arrow key 175 d. The enter key 176 is operatedby the operator, while the position of the cursor 176 is decided as theimage location Pn′ of the number “n” of the calibrating marker 106displayed on the number displaying section 177.

The calibration value of the second optical position information storedin the second optical position information storing section 130 is thencalculated by the calibration value calculating section 180 on the basisof the image location information extracted by the image locationinformation extracting section 170 and the estimated positioninformation stored in the estimated position information storing section150 (in the step S114).

The second optical position information stored in the second opticalposition information storing section 130 is finally calibrated by theoptical position information calibrating section 190 on the basis of thecalibration value calculated by the calibration value calculatingsection 180 (in the step S115). In this embodiment, the above mentionedsteps 101 to 115 are executed as a program by a computer.

From the above detail description, it will be understood that thepresent embodiment of the camera calibrating apparatus 100 according tothe present invention can calibrate the optical parameters of the camera110 mounted on the automotive vehicle 108 to ensure an accurate positionof an object located on the road is detected.

In this embodiment, the camera calibrating apparatus 100 according tothe present invention can calibrate the second optical position by usingthe simple calibrating marker 106.

In this embodiment, the camera calibrating apparatus 100 according tothe present invention can specify the image location of the calibratingmarker 106 to ensure that the image location information indicative ofthe image location of the calibrating marker 106 is extracted.

Second Embodiment

Referring now to the drawings, in particular to FIGS. 14 to 16, there isshown a second embodiment of a camera calibrating apparatus according tothe present invention.

The construction of the second embodiment of the camera calibratingapparatus according to the present invention will now be describedhereinafter. The constitutional elements of the second embodiment of thecamera calibrating apparatus the same as those of the first embodimentof the camera calibrating apparatus will not be described but bear thesame reference numerals and legends as those of the first embodiment ofthe camera calibrating apparatus.

In FIG. 14, the camera calibrating apparatus 200 comprises a calibratingsection 260 for calibrating the second optical position informationstored in the second optical position information storing section 130.

The calibrating section 260 is adapted to calibrate the second opticalposition information stored in the second optical position informationstoring section 130 on the basis of the image information of thecalibrating marker 106 obtained by the camera 110 and the estimatedposition information stored in the estimated position informationstoring section 150.

The calibrating section 260 includes an image location informationextracting section 270 for extracting image location informationindicative of an image location of the calibrating marker 106 to theimage coordinate system 114 of the camera 110 on the basis of the imageinformation of the calibrating marker 106 obtained by the camera 110, acalibration value calculating section 280 for calculating a calibrationvalue to the second optical position information stored in the secondoptical position information storing section 130 on the basis of theimage location information extracted by the image location informationextracting section 270 and the estimated position information stored inthe estimated position information storing section 150, and an opticalposition information calibrating section 290 for calibrating the secondoptical position information stored in the second optical positioninformation storing section 130 on the basis of the calibration valuecalculated by the calibration value calculating section 280.

The image location information extracting section 270 includes anestimated area information storing section 271 for storing estimatedarea information (see FIG. 15) indicative of an estimated area 274 ofthe calibrating marker 106 to the image coordinate system 114 of thecamera 110, and an image location searching section 272 for searchingthe image location Pn′ of the calibrating marker 106 from the imageinformation of the calibrating marker 106 obtained by the camera 110 onthe basis of the estimated area information stored by the estimated areainformation storing section 271 and the estimated position informationstored by the estimated location position information storing section150 to ensure that the image location information is extracted.

The calibration value calculating section 280 is adapted to calculatethe calibration value to the second optical position information on thebasis of a method which is the same as the method of the calibrationvalue calculating section 180 of the first embodiment.

The following description will be directed to the operation of thesecond embodiment of the camera calibrating apparatus according to thepresent invention. The second optical position information is calibratedby the camera calibrating apparatus 200 through the following stepsshown in FIG. 16.

The camera 110 is firstly located at the predetermined position on thefirst coordinate system 101 in a first working space (in the step S201).The first housing position information, the second housing positioninformation, the revising marker position information, and thecalibrating marker position information are then stored in the firsthousing position information storing section 115, the second housingposition information storing section 116, the revising marker positioninformation storing section 125, and the calibrating marker positioninformation storing section 126 (in the step S202), respectively. Here,the first housing position information, the second housing positioninformation, the revising marker position information, and thecalibrating marker position information are obtained on the basis ofpositions predetermined in the design process and measured by ameasuring instrument.

The image information of the revising marker 105 is then obtained by thecamera 110 (in the step S203). The first optical position information isthen produced by the first optical position information producingsection 117 from the revising marker position information stored in therevising marker position information storing section 125 on the basis ofthe image information of the revising marker 105 obtained by the camera110 (in the step S204). The first optical position information producedby the first optical position information producing section 117 is thenstored in the first optical position information storing section 118 (inthe step S205).

The second optical position information is then produced by the secondoptical position information producing section 120 from the secondhousing position information stored in the second housing positioninformation storing section 116 on the basis of the first housingposition information stored in the first housing position informationstoring section 115 and the first optical position information stored inthe first optical position information storing section 118 (in the stepS206). The second optical position information produced by the secondoptical position information producing section 120 is then stored in thesecond optical position information storing section 130 (in the stepS207).

The estimated position information indicative of the position of thecalibrating marker 106 to the image coordinate system of the camera 110is then produced by the estimated position information producing section140 from the calibrating marker position information stored in thecalibrating marker position information storing section 126 on the basisof the second optical position information produced by the secondoptical position information producing section 120 (in the step S208).The estimated position information produced by the estimated positioninformation producing section 140 is stored in the estimated positioninformation storing section 150 (in the step S209).

The camera 110 and the recordable medium 193 are then transferred to thesecond working space from the first working space. The camera 110 isthen mounted to the automotive vehicle 108 to be located at thepredetermined position on the second coordinate system 102 (in the stepS210).

The image information of the calibrating marker 106 is then obtained bythe camera 110 (in the step S211). As shown in FIG. 15, the imagelocation Pn′ is searched by the image location searching section 272 inthe estimated area 274 at the estimated location Pn of the calibratingmarker 106. The image location information is then extracted by theimage location information extracting section 270.

The calibration value to the second optical position information storedin the second optical position information storing section 130 is thencalculated by the calibration value calculating section 280 on the basisof the image location information extracted by the image locationinformation extracting section 270 and the estimated positioninformation stored in the estimated position information storing section150 (in the step S213).

The second optical position information stored in the second opticalposition information storing section 130 is finally calibrated by theoptical position information calibrating section 290 on the basis of thecalibration value calculated by the calibration value calculatingsection 280 (in the step S214). In this embodiment, the above mentionedsteps 101 to 115 may be executed as a program by a computer.

From the above detailed description, it will be understood that thesecond embodiment of the camera calibrating apparatus 200 can easilyextract the image location information of the calibrating marker 106 byreason that the image location of the calibrating marker 106 is searchedby the image location searching section 272.

Third Embodiment

Referring now to the drawings, in particular to FIGS. 17 to 28, there isshown a third embodiment of a camera calibrating apparatus according tothe present invention.

The construction of the third embodiment of the camera calibratingapparatus according to the present invention will now be describedhereinafter.

As shown in FIGS. 17 to 19, the camera calibrating apparatus 300 iselectrically connected to a camera 310 as an imaging device. The camera310 includes a housing unit 311 and an optical section 312 supported bythe housing unit 311 to obtain image information through the opticalsection 312.

The camera calibrating apparatus 300 comprises a first housing positioninformation storing section 315 for storing first housing positioninformation indicative of the position of the housing unit 311 to thefirst coordinate system 301, a second housing position informationstoring section 316 for storing second housing position informationindicative of the position of the housing unit 311 in the secondcoordinate system 302, and a revising marker position informationstoring section 325 for storing revising marker position informationindicative of the position of a revising marker 305 in the firstcoordinate system 301.

The first coordinate system 301 is defined in a first working space suchas a camera production plant to have X₁-axis, Y₁-axis, and Z₁-axis,while the revising marker 305 laid out in the first working space causesthe camera calibrating apparatus 300 to revise the camera 310 located inthe first working space. The revising marker 305 is constituted by aplurality of marking objects which are respectively located atpredetermined positions on the first coordinate system 301, and whichare within a viewing field of the camera 310 located in the firstworking space.

The second coordinate system 302 is defined in a second working spacesuch as an automotive vehicle production plant. The second coordinatesystem 302 has X₂-axis, Y₂-axis, and Z₂-axis. The X₂-axis and theY₂-axis collectively define X₂-Y₂ plane to be represented by a roadsurface 302 a on which an automotive vehicle 308 is located. Thedividing marker 307 is located on a bumper 309 of the automotive vehicle308. The dividing marker 307 is constituted by two marking objects whichis within a viewing field of the camera 310 mounted on the automotivevehicle 308. The dividing marker 307 is located directly below thecamera 310 mounted on the automotive vehicle 308.

The camera calibrating apparatus 300 is adapted to revise the camera 310in the first working space. The camera 310 is located at a predeterminedposition on the first coordinate system 301, while the first housingposition information storing section 315 is adapted to store firsthousing position information indicative of the position of the housingunit 311. Here, the term “the calibration of the camera 310” is intendedto indicate the calculation of the accurate position of the opticalsection 312 of the camera 310 mounted on the automotive vehicle 308.

The camera 310 revised by the camera calibrating apparatus 310 ismounted on the automotive vehicle 308 to be located at a predeterminedposition on the second coordinate system 302 in the second workingspace, while the second housing position information storing section 316is adapted to store second housing position information indicative ofthe position of the housing unit 311. Here, the second housing positioninformation indicates an accurate position in which the housing unit 311of the camera 310 is located when the camera 310 is mounted on theautomotive vehicle 308.

The dividing marker 307 is located on the bumper 309 of the automotivevehicle 308 in spaced relationship with the position of the housing unit311 represented by the second housing position information. This meansthat the dividing marker 307 is not moved with respect to the housingunit 311 when the automotive vehicle 308 is traveling on the road 302 a.

The camera calibrating apparatus 300 further comprises a first opticalposition information producing section 317 for producing first opticalposition information indicative of the position of the optical section312 to the first coordinate system 301, and a first optical positioninformation storing section 318 for storing the first optical positioninformation produced by the first optical position information producingsection 317.

The first optical position information producing section 317 is adaptedto calculate the position of the optical section 312 to the firstcoordinate system 301 from the revising marker position informationstored in the revising marker position information storing section 325on the basis of the image information of the revising marker 305obtained by the camera 310. Here, the term “the position of the opticalsection 312” is intended to indicate an optical center of the opticalsection 312 and the position of an optical axis, and other opticalparameters. The calculation of the position of the optical section 312to the first coordinate system 301 is performed on the basis of themethod disclosed in the document 1.

The camera calibrating apparatus 300 further comprises a second opticalposition information producing section 320 for producing second opticalinformation indicative of the position of the optical section 312 in thesecond coordinate system 302, and a second optical position informationstoring section 330 for storing the second optical position informationproduced by the second optical position information producing section320.

The second optical position information producing section 320 is adaptedto produce second optical position information indicative of theposition of the optical section 312 in the second coordinate system 302from the second housing position information stored in the secondhousing position information storing section 316 on the basis of thefirst housing position information stored by the first housing positioninformation storing section 315 and the first optical positioninformation stored in the first optical position information storingsection 318.

The second optical position information producing section 320 is adaptedto calculate the position of the optical section 312 in the secondcoordinate system 302 on the basis of a method which is the same as themethod of the second optical position information producing section 120of the first embodiment.

As shown in FIG. 20, the camera 310 mounted on the automotive vehicle308 located in the second working space defines a camera coordinatesystem 313 having x-axis, y-axis, and z-axis on the basis of the secondoptical position information. The origin point of the camera coordinatesystem 313 is in register with the optical center of the optical section312. The x-axis of the camera coordinate system 313 extends in ahorizontal direction of the camera 310. The y-axis of the cameracoordinate system 313 extends in a vertical direction of the camera 310.The z-axis of the camera coordinate system 313 is in axial alignmentwith the optical axis of the optical section 312.

The image coordinate system 314 is defined on a plane spaced apart fromthe origin point of the camera coordinate system 313 with a focal length“f” in the direction of the z-axis of the camera coordinate system 313.The image coordination system 314 has p-axis and q-axis. The camera 310is adapted to obtain, as image information, an optical image focused onthe image coordinate system 314 through the optical section 312.

The camera calibrating apparatus 300 further comprises a calibratingsection 360 for calibrating the second optical position informationstored in the second optical position information storing section 330 onthe basis of the motion vector of the image information obtained by thecamera 310 in the second coordinate system 302.

The calibrating section 360 includes a plane-projected image producingsection 361 for producing a plane-projected image from the imageinformation obtained by the camera 310 on the second coordinate system302, a plane-projected image dividing section 362 for dividing theplane-projected image produced by the plane-projected image producingsection 361 into a plurality of image segments, a motion vectorextracting section 363 for extracting the motion vector from the imagesegments divided by the plane-projected image dividing section 362, acalibration value calculating section 364 for calculating a calibrationvalue of the second optical position information stored by the secondoptical position information storing section 330 on the basis of themotion vector extracted by the motion vector extracting section 363, andan optical position information calibrating section 365 for calibratingthe second optical position information stored by the second opticalposition information storing section 330 on the basis of the calibrationvalue calculated by the calibration value calculating section 364.

As shown in FIG. 21, the plane-projected image produced by theplane-projected image producing section 361 is the sama as an imagetaken by an imaginary camera 370 located on a plane which defines animage coordinate system 371 parallel to the road surface 302 a. Theimage of the road surface 302 a focused on the image coordinate system371 is the same as the reduced-size road surface 302 a.

The plane-projected image producing section 361 is adapted to produceplane-projected image on the basis of a following method.

In FIG. 20, the angle of the x-axis of the image coordinate system 313to the road surface 302 a is represented by a legend α. The angle of they-axis of the image coordinate system 313 to the road surface 302 a isrepresented by a legend β. The distance between the origin point of thecamera coordinate system 313 and an intersecting point of the extensionline of the z-axis of the camera coordinate system 313 to the roadsurface 302 a is represented by a legend “c”. The X₂-Y₂ surface of thesecond coordinate system 302, i.e., the road surface 302 a isrepresented by following equations (6) and (7).

$\begin{matrix}\left\{ \begin{matrix}{a = {\sin\;\alpha}} \\{b = {\sin\;\beta}}\end{matrix} \right. & (6)\end{matrix}$z=αx+by+c  (7)

The transformation to the camera coordinate system 313 from the imagecoordinate system 314 is represented by a following equation (8). Here,the legend (p, q) represents ordinate and abscissa of the image locationP′ to the image coordinate system 314.

$\begin{matrix}\left\{ \begin{matrix}{x = \frac{cp}{f - {ap} - {bp}}} \\{y = \frac{cq}{f - {ap} - {bp}}} \\{z = \frac{cf}{f - {ap} - {bp}}}\end{matrix} \right. & (8) \\{\begin{bmatrix}X \\Y \\Z\end{bmatrix} = {{\begin{bmatrix}R_{11} & R_{12} & R_{13} \\R_{21} & R_{22} & R_{23} \\R_{31} & R_{32} & R_{33}\end{bmatrix}\begin{bmatrix}x \\y \\z\end{bmatrix}} + \begin{bmatrix}T_{x} \\T_{y} \\T_{z}\end{bmatrix}}} & (9)\end{matrix}$

In the equation (9), the translation vector “T” indicates a direction ofthe origin point of the second coordinate system 302 to the origin pointof the camera coordination system 313 and a distance between the originpoint of the second coordinate system 302 and the origin point of thecamera coordinate system 313. The rotation matrix “R” represents adeviation between the second coordinate system 302 and the cameracoordinate system 313 in the rotational direction. When the housing unit311 is mounted on the slanted position on the automotive vehicle 308,the rotation matrix “R” is represented by the equations (3) to (5) shownin the first embodiment. Here, the legends θ, Φ, and φ represent theangle of the camera coordinate system 313 to the x-axis of the cameracoordinate system 313, the angle of the camera coordinate system 313 tothe y-axis of the camera coordinate system 313, and the angle of thecamera coordinate system 313 to the z-axis of the camera coordinatesystem 313, respectively.

When the housing unit 311 is mounted on the accurate position on theautomotive vehicle 308, the plane-projected image producing section 361is adapted to produce a plane-projected image shown in FIG. 22( b) fromthe image information shown in FIG. 22( a). In this plane-projectedimage, the lines 373 and 374 are in parallel relationship with eachother by reason that the plane-projected image is the same as the imagetaken by the imaginary camera 370 located in parallel relationship withthe road surface 302 a.

When the camera coordinate system 313 is moved around the x-axis underthe condition that the housing unit 311 is downwardly slanted withrespect to the automotive vehicle 308, the plane-projected imageproducing section 361 is adapted to produce a plane-projected imageshown in FIG. 23( b) from the image information shown in FIG. 23( a).

When the camera coordinate system 313 is moved around the y-axis underthe condition that the housing unit 311 is leftwardly panned withrespect to the automotive vehicle 308, the plane-projected imageproducing section 361 is adapted to produce a plane-projected imageshown in FIG. 24( b) from the image information shown in FIG. 24( a).

When the camera coordinate system 313 is moved around the z-axis underthe condition that the housing unit 311 is leftwardly slanted withrespect to the automotive vehicle 308, the plane-projected imageproducing section 361 is adapted to produce a plane-projected imageshown in FIG. 25( b) from the image information shown in FIG. 25( a).

As shown in FIGS. 22( b), 23(b), 24(b), and 25(b), the plane-projectedimage dividing section 362 is adapted to divide the plane-projectedimage into a plurality of image segments on the basis of the imageinformation of the dividing markers 307 obtained by the camera 310.

The plane-projected image dividing section 362 is adapted to superimposedividing lines 383 and 384 on the plane-projected image on the basis ofa base line 380 passing through the dividing markers 307. The dividingline 383 is in perpendicular relationship with the base line 380, andpasses thought an intermediate point 381 defined between the dividingmarkers 307. On the other hand, the dividing line 384 is in parallel andspaced relationship with the base line 380.

When the housing unit 311 is mounted on the slanted position of theautomotive vehicle 308, more specifically, the camera coordinate system313 is moved around the z-axis, the base line 380 has a tilt to theright as will be seen from the plane-projected image shown in FIG. 25(b).

As shown in FIGS. 22( b), 23(b), 24(b), and 25(b), the motion vectorextracting section 363 is adapted to extract the motion vectors 388 a,388 b, 388 c, and 388 d from the four image segments 386 a, 386 b, 386c, and 386 d divided by the plane-projected image dividing section 362.

The directions of the motion vectors 388 a to 388 d are equal to oneanother when the automotive vehicle 308 is linearly traveling by reasonthat the motion vectors 388 a to 388 d are calculated on the basis ofthe flow of the fragmental image of the image information obtained fromthe motion of the automotive vehicle 308, i.e., the motion of the camera310 to the road. The following description will be directed to themotion vectors 388 a to 388 d extracted from the image informationobtained under the automotive vehicle 308 is linearly traveling.

When the housing unit 311 is mounted on the accurate position of theautomotive vehicle 308, the amplitudes of the motion vectors 388 a, 388b, 388 c, and 388 d are equal to one another as shown in FIG. 22( b).

When the housing unit 311 is mounted on the downwardly slanted positionof the automotive vehicle 308, more specifically, the camera coordinatesystem 313 is moved around the x-axis, the motion vectors 388 a and 338b of the upper side fragmental image is larger than the motion vectors388 a and 338 b of the lower side fragmental image as shown in FIG. 23(b).

When the housing unit 311 is mounted on the leftwardly slanted positionof the automotive vehicle 308, more specifically, the camera coordinatesystem 313 is moved around the y-axis, the motion vectors 388 b and 338d of the right side fragmental image is larger than the motion vectors388 a and 338 c of the left side fragmental image as shown in FIG. 24(b).

The calibration value calculating section 364 is adapted to produce acalibration value of the second optical position information on thebasis of a following method.

As shown in FIGS. 22( c), 23(c), and 24(c), the vector 390 a equal inamplitude to the motion vector 388 a, the vector 390 b equal inamplitude to the motion vector 388 b, the vector 390 c equal inamplitude to the motion vector 388 c, and the vector 390 d equal inamplitude to the motion vector 388 d are located on a composited vectorcalculating coordinate system 391.

Here, the vector 390 a is located at the origin point of the compositedvector calculating coordinate system 391 in the direction of anupper-leftward angle of 45 degrees. The vector 390 b is located at theorigin point of the composited vector calculating coordinate system 391in the direction of an upper-rightward angle of 45 degrees. The vector390 c is located at the origin point of the composited vectorcalculating coordinate system 391 in the direction of a lower-leftwardangle of 45 degrees. The vector 390 a is located at the origin point ofthe composited vector calculating coordinate system 391 in the directionof a lower-rightward angle of 45 degrees.

The composition of the vectors 390 a to 390 d is performed on thecomposited vector calculating coordinate system 391.

When the housing unit 311 is mounted on the accurate position of theautomotive vehicle 308, the amplitude of the composited vector 392 isequal to zero by reason that the amplitudes of the motion vectors 388 a,388 b, 388 c, and 388 d are equal to one another as shown in FIG. 22(b). Here, each of the rotation angle θ to the x-axis, the rotation angleΦ to the y-axis, and the rotation angle φ to the z-axis is equal to zeroon the camera coordinate system 313.

When the housing unit 311 is mounted on the downwardly slanted positionof the automotive vehicle 308, and more specifically, the cameracoordinate system 313 is moved around the x-axis, the composited vector392 points upward by reason that the vectors 390 a and 390 b arerespectively larger in amplitude than the vectors 390 a and 390 b asshown in FIG. 23( b). The rotation angle θ to the x-axis is calculatedon the basis of the equation (9) under the condition that the calculatedrotation matrix “R” leads to the fact that the composited vector 392 isequal to zero.

When the housing unit 311 is mounted on the leftwardly slanted positionof the automotive vehicle 308, and more specifically, the cameracoordinate system 313 is moved around the y-axis, the composited vector392 points rightward by reason that the vectors 390 b and 390 d arerespectively larger in amplitude than the vectors 390 a and 390 c asshown in FIG. 24( b). The rotation angle Φ to the y-axis is calculatedon the basis of the equation (9) under the condition that the calculatedrotation matrix “R” leads to the fact that the composited vector 392 isequal to zero.

When, on the other hand, the housing unit 311 is mounted on the slantedposition of the automotive vehicle 308, and more specifically, thecamera coordinate system 313 is moved around the z-axis, the rotationangle φ to the z-axis of the camera coordinate system 313 is calculatedon the basis of the slope of the base line 380 on the image coordinatesystem 371 of the plane-projected image shown in FIG. 25( b).

As shown in FIG. 26, the camera calibrating apparatus 300 thusconstructed as previously mentioned can be realized by comprising acomputer 391 for adjusting the camera 310, ECU 392 constituted as animage controlling apparatus for controlling the camera 310, and otherelements.

The computer 391 is shown in FIG. 26( a) as including CPU, RAM, ROM, aninput/output interface, and other units. The computer 391 iselectrically connected to the camera 310 at the first working space. Inthis embodiment, the computer 391 constitutes each of the first housingposition information storing section 315, the second housing positioninformation storing section 316, the first optical position informationproducing section 317, the first optical position information storingsection 318, the second optical position information producing section320, the revising marker position information storing section 325, andthe second optical position information storing section 330.

The ECU 392 is shown in FIG. 26( c) as including CPU, RAM, ROM, aninput/output interface, and other elements. The ECU 392 is mounted onthe automotive vehicle 308 to be electrically connected to the camera310 in the second working space. In this embodiment, the ECU 392constitutes each of the second optical position information storingsection 330 and the calibrating section 360.

As shown in FIG. 26( b), the camera 310 is transferred to the secondworking space from the first working space with a recordable medium 393such as for example CD-ROM and a magnetic disk. The recordable medium393 has stored therein the second optical position information to ensurethat the second optical position information is effectively transferredto the ECU 392 from the computer 391.

In this embodiment, the camera 310 and the recordable medium 393 aretransferred to the second working space from the first working space,however, the camera 310, the second optical position information storingsection 330, and the calibrating section 360 may be collectivelytransferred as a camera unit 194 to the second working space from thefirst working space as shown in FIG. 27.

The following description will be directed to the operation of the thirdembodiment of the camera calibrating apparatus according to the presentinvention.

The second optical position information is calibrated by the cameracalibrating apparatus 300 through the steps shown in the FIG. 28.

The camera 310 is firstly located at the predetermined position on thefirst coordinate system 301 in the first working space (in the stepS301). The first housing position information, the second housingposition information, and the revising marker position information arethen stored in the first housing position information storing section315, the second housing position information storing section 316, andthe revising marker position information storing section 325,respectively (in the step S302). Here, the first housing positioninformation, the second housing position information, and the revisingmarker position information are obtained on the basis of positionspredetermined in the design process and measured by a measuringinstrument.

The image information of the revising marker 105 is then obtained by thecamera 310 (in the step S303). The first optical position information isthen produced by the first optical position information producingsection 317 from the revising marker position information stored in therevising marker position information storing section 325 on the basis ofthe image information of the revising marker 305 obtained by the camera310 (in the step S304). The first optical position information producedby the first optical position information producing section 317 is thenstored in the first optical position information storing section 318 (inthe step S305).

The second optical position information is then produced by the secondoptical position information producing section 320 from the secondhousing position information stored in the second housing positioninformation storing section 316 on the basis of the first housingposition information stored in the first housing position informationstoring section 315 and the first optical position information stored inthe first optical position information storing section 318 (in the stepS306). The second optical position information produced by the secondoptical position information producing section 320 is then stored in thesecond optical position information storing section 330 (in the stepS307).

The camera 310 and the recordable medium 393 are then transferred to thesecond working space from the first working space. The camera 310 isthen mounted to the automotive vehicle 308 to be located at apredetermined position on the second coordinate system 302 (in the stepS308).

The image of the road surface 302 a is then taken by the camera 310 (inthe step S309). As shown in FIGS. 22( b), 23(b), 24(b), and 25(b), theplane-projected image is produced from the image information obtained bythe camera 310 on the second coordinate system 302 (in the step S310).

As shown in FIGS. 22( b), 23(b), 24(b), and 25(b), the plane-projectedimage produced by the plane-projected image producing section 361 isdivided into a plurality of image segments 386 a to 386 d by theplane-projected image dividing section 362 on the basis of the imageinformation of the dividing markers 307 obtained by the camera 310 (inthe step S311).

As shown in FIGS. 22( b), 23(b), 24(b), and 25(b), the motion vectorsare extracted by the motion vector extracting section 363 from the imagesegments 386 a to 386 d divided by the plane-projected image dividingsection 362 (in the step S312).

As shown in FIGS. 25( b), the calibration value on z-axis of the secondoptical position information stored in the second optical positioninformation storing section 330 is then calculated by the calibrationvalue calculating section 365 on the basis of the slope of the base line380 on the image coordinate system 371 of the plane-projected imageshown in FIG. 25( b) (in the step S313). As shown in FIGS. 22( c),23(c), and 24(c), the calibration value on each of x-axis and y-axis ofthe second optical position information stored in the second opticalposition information storing section 330 is then calculated by thecalibration value calculating section 365 on the basis of the motionvectors extracted by the motion vector extracting section 363 (in thestep S314).

The second optical position information stored in the second opticalposition information storing section 330 is finally calibrated by theoptical position information calibrating section 365 on the basis of thecalibration value calculated by the calibration value calculatingsection 364 (in the step S315). In this embodiment, the above mentionedsteps 301 to 315 may be executed as a program by a computer.

From the above detailed description, it will be understood that thecamera calibrating apparatus 300 can calibrate the second opticalposition information by using the motion vectors 388 a to 388 d.

In this embodiment, the camera calibrating apparatus 300 can easilyextract the motion vectors from the respective image segments 386 a to386 d.

In this embodiment, the camera calibrating apparatus 300 can accuratelydivide the plane-projected image by using the dividing marker 307.

Fourth Embodiment

Referring now to the drawings, in particular to FIGS. 29 to 38, there isshown a fourth embodiment of a camera calibrating apparatus according tothe present invention.

The construction of the fourth embodiment of the camera calibratingapparatus according to the present invention will now be describedhereinafter.

As shown in FIGS. 29 to 31, the camera calibrating apparatus 400 iselectrically connected to a camera 410 as an imaging device. The camera410 includes a housing unit 411 and an optical section 412 supported bythe housing unit 411 to obtain image information through the opticalsection 412.

The camera calibrating apparatus 400 comprises a first housing positioninformation storing section 415 for storing first housing positioninformation indicative of the position of the housing unit 411 to thefirst coordinate system 401, a second housing position informationstoring section 416 for storing second housing position informationindicative of the position of the housing unit 411 in the secondcoordinate system 402, a revising marker position information storingsection 425 for storing revising marker position information indicativeof the position of a revising marker 405 to the first coordinate system401, and a vehicle position information storing section 426 for storingvehicle position information indicative of the position of, for example,a rear bumper 409 of an automotive vehicle 408 in the second coordinatesystem 402.

The first coordinate system 401 is defined in a first working space suchas a camera production plant to have X₁-axis, Y₁-axis, and Z₁-axis. Therevising marker 405 laid out in the first working space causes thecamera calibrating apparatus 400 is revise the camera 410 located in thefirst working space. The revising marker 405 is constituted by aplurality of marking objects which are respectively located atpredetermined positions on the first coordinate system 401, and whichare within a viewing field of the camera 410 located in the firstworking space.

The second coordinate system 402 is defined in a second working spacesuch as an automotive vehicle production plant to have X₂-axis, Y₂-axis,and Z₂-axis, while X₂-axis and Y₂-axis collectively define X₂-Y₂ planewhich is represented by a road surface 402 a on which an automotivevehicle 408 is located. The calibrating plate 406 painted in a singlecolor is located on the road surface 402 a. The monochromaticcalibrating plate 406 located directly below the automotive vehicle 408is different in brightness, chromaticity, and saturation of color fromthe automotive vehicle 408.

The camera calibrating apparatus 400 is adapted to revise the camera 410in the first working space. The camera 410 is located at a predeterminedposition on the first coordinate system 401, while the first housingposition information storing section 415 is adapted to store firsthousing position information indicative of the position of the housingunit 411. Here, the term “the calibration of the camera 410” is intendedto indicate the calculation of the accurate position of the opticalsection 412 of the camera 410 mounted on the automotive vehicle 408.

The camera 410 revised by the camera calibrating apparatus 410 ismounted on the automotive vehicle 408 to be located at a predeterminedposition on the second coordinate system 402 in the second workingspace, while the second housing position information storing section 416is adapted to store second housing position information indicative ofthe position of the housing unit 411. Here, the second housing positioninformation indicates an accurate position in which the housing unit 411of the camera 410 is located when the camera 410 is mounted on theautomotive vehicle 408.

The camera calibrating apparatus 400 further comprises a first opticalposition information producing section 417 for producing first opticalposition information indicative of the position of the optical section412 to the first coordinate system 401, and a first optical positioninformation storing section 418 for storing the first optical positioninformation produced by the first optical position information producingsection 417.

The first optical position information producing section 417 is adaptedto calculate the position of the optical section 412 to the firstcoordinate system 401 from the revising marker position informationstored in the revising marker position information storing section 425on the basis of the image information of the revising marker 305obtained by the camera 410. Here, the term “the position of the opticalsection 412” is intended to indicate an optical center of the opticalsection 412 and the position of an optical axis, and other opticalparameters. The calculation of the position of the optical section 412to the first coordinate system 401 is performed on the basis of themethod disclosed in the document 1.

The camera calibrating apparatus 400 further comprises a second opticalposition information producing section 420 for producing second opticalinformation indicative of the position of the optical section 412 in thesecond coordinate system 402, and a second optical position informationstoring section 430 for storing the second optical position informationproduced by the second optical position information producing section420.

The second optical position information producing section 420 is adaptedto produce second optical position information indicative of theposition of the optical section 412 in the second coordinate system 402from the second housing position information stored in the secondhousing position information storing section 416 on the basis of thefirst housing position information stored by the first housing positioninformation storing section 415 and the first optical positioninformation stored in the first optical position information storingsection 418.

The second optical position information producing section 420 is adaptedto calculate the position of the optical section 412 in the secondcoordinate system 402 on the basis of a method which is the same as themethod of the second optical position information producing section 120of the first embodiment.

As shown in FIG. 32, the camera 410 mounted on the automotive vehicle408 in the second working space defines a camera coordinate system 413having x-axis, y-axis, and z-axis on the basis of the second opticalposition information. The origin point of the camera coordination system413 is in register with the optical center of the optical section 412.The x-axis of the camera coordinate system 413 extends in a horizontaldirection of the camera 410. The y-axis of the camera coordinate system413 extends in a vertical direction of the camera 410. The z-axis of thecamera coordinate system 413 is in axial alignment with the optical axisof the optical section 412.

The image coordinate system 414 is defined on a plane spaced apart fromthe origin point of the camera coordinate system 413 with a focal length“f” in the direction of the z-axis of the camera coordinate system 413.The image coordination system 414 has p-axis and q-axis. The camera 410is adapted to obtain, as image information, an optical image focused onthe image coordinate system 414 through the optical section 412.

The camera calibrating apparatus 400 further comprises an estimatedposition information producing section 440 for producing estimatedposition information of the bumper 409 of the automotive vehicle 408 tothe image coordinate system 414 of the camera 410, and an estimatedposition information storing section 450 for storing the estimatedposition information produced by the estimated position informationproducing section 440.

The estimated position information producing section 440 is adapted toproduce estimated position information of the calibrating marker 406 tothe image coordinate system 414 of the camera 410 from the positioninformation of the calibrating marker 406 stored in the vehicle positioninformation storing section 426 on the basis of the second opticalposition information produced by the second optical position informationproducing section 420. The calculation of the estimated location of thecalibrating marker 406 to the image coordinate system 414 of the camera410 is performed on the basis of the above mentioned document 1.

As shown in FIG. 33, the bumper 409 of the automotive vehicle 408located on the second coordinate system 402 is focused on the imagelocation Pn′ of the image coordinate system 414 through the opticalsection 412. Here, the image location Pn′ is in register with theestimated image location Pn calculated by the estimated image locationcalculating section 440 under the condition that the housing unit 411 islocated on an accurate position of the automotive vehicle 408, morespecifically, the housing unit 411 is located at the positionrepresented by the second housing position information with no deviationto the accurate position. However, the housing unit 411, in general, islocated at an inaccurate position with deviation to the accurateposition. This leads to the fact that the second optical positioninformation is produced with deviation to the accurate position. As aresult, the estimated location Pn is spaced apart from the imagelocation Pn′ of the image coordinate system 414.

The camera calibrating apparatus 400 further comprises a calibratingsection 460 for calibrating the second optical position informationstored in the second optical position information storing section 430 toorder to calibrate the above mentioned deviation of the second opticalposition information.

The calibrating section 460 is adapted to calibrate the second opticalposition information stored in the second optical position informationstoring section 430 on the basis of the image information of the bumper409 obtained by the camera 410 and the estimated position informationstored in the estimated position information storing section 450.

The calibrating section 460 includes an image location informationextracting section 470 for extracting image location informationindicative of an image location of the bumper 409 to the imagecoordinate system 414 of the camera 410 on the basis of the imageinformation of the calibrating marker 406 obtained by the camera 110, acalibration value calculating section 480 for calculating a calibrationvalue of the second optical position information stored in the secondoptical position information storing section 430 on the basis of theimage location information extracted by the image location informationextracting section 470 and the estimated position information stored inthe estimated position information storing section 450, and an opticalposition information calibrating section 490 for calibrating the secondoptical position information stored in the second optical positioninformation storing section 430 on the basis of the calibration valuecalculated by the calibration value calculating section 480.

The calibration value calculating section 480 includes a superimposingsection 481 for superimposing one of a profile line of the bumper 409 ofthe image location information and a profile line of the bumper 409 ofthe estimated location information on the other of the profile line ofthe bumper 409 of the image location information and the profile line ofthe bumper 409 of the estimated location information, an extractingsection 482 for extracting two or more points, for example points of endportions of the bumper, from the profile lines of the automotive vehiclesuperimposed by the superimposing section 481, and a calculating section483 for calculating the calibration value of the second optical positioninformation stored by the second optical position information bycomparing the points of the image location information with the pointsof the estimated location information.

The superimposing section 481 is adapted to superimpose the profile line“P” (see FIG. 34( a)) of the bumper 409 of the estimated locationinformation on the profile line “P” (see FIG. 34( b)) of the bumper 409of the image location information by allowing the profile line “P′” ofthe bumper 409 of the estimated location information to be linearly androtationally moved with respect to the profile line “P′” of the bumper409 of the image location information (see FIG. 34( c)).

As shown in FIG. 35( a), the extracting section 482 is adapted toextract two or more points (P₁, P₂) and (P₁′, P₂′) from the profilelines “P” and “P′” of the bumper 409 superimposed by the superimposingsection 481.

The calculating section 483 is adapted to calculate a calibration valueof the second optical position information on the basis of a methodwhich is the same as the method of the calibration value calculatingsection 180 in the first embodiment by comparing the points (P₁, P₂)(see FIG. 35( b)) of the estimated location information with the points(P₁′, P₂′) (see FIG. 35( b)) of the image location information.

As shown in FIG. 36, the camera calibrating apparatus 400 thusconstructed as previously mentioned can be realized by comprising acomputer 491 for adjusting the camera 410, ECU 492 constituted as animage controlling apparatus for controlling the camera 410, and otherelements.

The computer 491 is shown in FIG. 26( a) as including CPU, RAM, ROM, aninput/output interface, and other elements. The computer 491 iselectrically connected to the camera 410 at the first working space. Inthis embodiment, the computer 491 constitutes each of the first housingposition information storing section 415, the second housing positioninformation storing section 416, the first optical position informationproducing section 417, the first optical position information storingsection 418, the second optical position information producing section420, the calibrating marker position information storing section 425,the calibrating marker position information storing section 426, thesecond optical position storing section 430, the estimated positioninformation producing section 440, and the estimated positioninformation storing section 450.

The ECU 492 is shown in FIG. 36( c) as including CPU, RAM, ROM, aninput/output interface, and other elements. The ECU 492 is mounted onthe automotive vehicle 408 to be electrically connected to the camera410 in the second working space. In this embodiment, the ECU 492constitutes each of the second optical position information storingsection 430, the estimated position information storing section 450, andthe calibrating section 460.

As shown in FIG. 36( b), the camera 410 is transferred to the secondworking space from the first working space with a recordable medium 493such as for example CD-ROM and a magnetic disk. The recordable medium493 has stored therein each of the second optical position informationand the estimated location information to ensure that the each of thesecond optical position information and the estimated locationinformation is effectively transferred to the ECU 492 from the computer491.

In this embodiment, the camera 410 and the recordable medium 493 aretransferred to the second working space from the first working space,however, the camera 410, the second optical position information storingsection 430, and the calibrating section 460 may be collectivelytransferred as a camera unit 494 to the second working space from thefirst working space as shown in FIG. 37.

The following description will be directed to the operation of thefourth embodiment of the camera calibrating apparatus according to thepresent invention.

The second optical position information is calibrated by the cameracalibrating apparatus 400 through the steps shown in the FIG. 38.

The camera 410 is firstly located at the predetermined position on thefirst coordinate system 401 in the first working space (in the stepS401). The first housing position information, the second housingposition information, the revising marker position information, and thevehicle position information are then stored in the first housingposition information storing section 415, the second housing positioninformation storing section 416, the revising marker positioninformation storing section 425, and the vehicle position informationstoring section 426, respectively (in the step S402). Here, the firsthousing position information, the second housing position information,the revising marker position information, and the vehicle positioninformation are obtained on the basis of positions predetermined in thedesign process and measured by a measuring instrument.

The image information of the revising marker 405 is then obtained by thecamera 410 (in the step S403). The first optical position information isthen produced by the first optical position information producingsection 417 from the revising marker position information stored in therevising marker position information storing section 425 on the basis ofthe image information of the revising marker 405 obtained by the camera410 (in the step S404). The first optical position information producedby the first optical position information producing section 417 is thenstored in the first optical position information storing section 418 (inthe step S405).

The second optical position information is then produced by the secondoptical position information producing section 420 from the secondhousing position information stored in the second housing positioninformation storing section 416 on the basis of the first housingposition information stored in the first housing position informationstoring section 415 and the first optical position information stored inthe first optical position information storing section 418 (in the stepS406). The second optical position information produced by the secondoptical position information producing section 420 is then stored in thesecond optical position information storing section 430 (in the stepS407).

The estimated location information indicative of the position of thebumper 409 to the image coordinate system 414 of the camera 410 is thenproduced by the estimated position information producing section 440from the vehicle position information stored in the vehicle positioninformation storing section 426 on the basis of the second opticalposition information produced by the second optical position informationproducing section 420 (in the step S408). The estimated positioninformation produced by the estimated position information producingsection 440 is stored in the estimated position information storingsection 450 (in the step S409).

The camera 410 and the recordable medium 493 are then transferred to thesecond working space from the first working space. The camera 410 isthen mounted to the automotive vehicle 408 to be located at thepredetermined position on the second coordinate system 402 (in the stepS410).

The image of the bumper 409 is then taken by the camera 310 against thecalibrating plate 406 (in the step S411). As shown in FIG. 33, the imagelocation information of the bumper 409 is extracted from the imageinformation of the bumper 409 obtained by the camera 410 on the imagecoordinate system 414 of the camera 410 (in the step S412).

As shown in FIG. 34, the profile line P′ of the bumper 409 of the imagelocation information is superimposed on the profile line P of the bumper409 of the estimated image information by the superimposing section 481(in the step S413).

As shown in FIG. 35, two or more points (P₁, P₂) and (P₁′, P₂′) of theoverlapped portion are then extracted from the profile lines “P” and“P′” of the bumper 409 by the extracting section 482 (in the step S414).The calibration value of the second optical position information is thencalculated by the calculating section 483 by comparing the points (P₁′,P₂′) of the image location information with the points (P₁, P₂) of theestimated location information (in the step S415).

The second optical position information stored in the second opticalposition information storing section 430 is finally calibrated by theoptical position information calibrating section 490 on the basis of thecalibration value calculated by the calibration value calculatingsection 480 (in the step S416). In this embodiment, the above mentionedsteps 401 to 415 may be executed as a program by a computer.

From the above detailed description, it will be understood that thecamera calibrating apparatus 400 can calibrate the second opticalposition information by using the motion vectors 388 a to 388 d.

In this embodiment, the camera calibrating apparatus 400 can calibratethe second optical position information by using the bumper 409 formingpart of the automotive vehicle 408.

In this embodiment, the camera calibrating apparatus 400 can accuratelydivide the plane-projected image by using the dividing marker 307.

As will be seen from the above detailed description, the cameracalibrating apparatus according to the present invention can calibratethe optical parameter of the camera mounted to the automotive vehicle.

1. A camera calibrating apparatus to be operative in combination with animaging device which includes a housing unit and an optical sectionsupported by said housing unit to obtain image information through saidoptical section, and adapted to calibrate optical position informationindicative of a position of said optical section, comprising: firsthousing position information storing means for storing first housingposition information indicative of a position of said housing unit inthe first coordinate system in which a revising marker is located;second housing position information storing means for storing secondhousing position information indicative of a position of said housingunit in the second coordinate system in which a calibrating marker islocated; first optical position information calculating means forcalculating first optical position information indicative of a positionof said optical section in said first coordinate system on the basis ofsaid image information of said revising marker obtained by said imagingdevice; first optical position information storing means for storingsaid first optical position information produced by said first opticalposition information producing means; second optical positioninformation calculating means for calculating second optical positioninformation indicative of a position of said optical section to saidsecond coordinate system from said second housing position informationstored by said second housing position information storing means on thebasis of said first housing position information stored by said firsthousing position information storing means and said first opticalposition information stored by said first optical position informationstoring means; estimated position information calculating means forcalculating estimated position information indicative of a position ofsaid calibrating marker to an image coordinate system of said imagingdevice on the basis of said second optical position information producedby said second optical position information producing means; secondoptical position information storing means for storing said secondoptical position information produced by said second optical positioninformation producing means; estimated position information storingmeans for storing said estimated position information produced by saidestimated position information producing means; and calibrating meansfor calibrating said second optical position information stored by saidsecond optical position information storing means on the basis of saidimage information of said calibrating marker obtained by said imagingdevice and said estimated position information stored by said estimatedposition information storing means.
 2. A camera calibrating apparatus asset forth in claim 1, in which said calibrating means includes: imagelocation information extracting means for extracting an image locationinformation indicative of an image location of said calibrating markerto said image coordinate system of said imaging device on the basis ofsaid image information of said calibrating marker obtained by saidimaging device; calibration value calculating means for calculating acalibration value of said second optical position information stored bysaid second optical position information storing means on the basis ofsaid image location information extracted by said image locationinformation extracting means and said estimated position informationstored by said estimated position information storing means; and opticalposition information calibrating means for calibrating said secondoptical position information stored by said second optical positioninformation storing means on the basis of said calibration valuecalculated by said calibration value calculating means.
 3. A cameracalibrating apparatus as set forth in claim 2, in which said calibratingmeans is adapted to calibrate a deviation of a rotational component ofsaid second optical position information.
 4. A camera calibratingapparatus as set forth in claim 2, in which said image locationinformation extracting means includes: image displaying means fordisplaying an image of said calibrating marker obtained by said imagingdevice, and image location specifying means for specifying said imagelocation of said calibrating marker on said image of said calibratingmarker displayed by said image displaying means and to ensure that saidimage location information is extracted.
 5. A camera calibratingapparatus as set forth in claim 2, in which said image locationinformation extracting means includes: estimated area informationstoring means for storing estimated area information indicative of saidcalibrating marker to said image coordinate system of said imagingdevice; and image location searching means for searching said imagelocation of said calibrating marker from said image information of saidcalibrating marker obtained by said imaging device on the basis of saidestimated area information stored by said estimated area informationstoring means and said estimated position information stored by saidestimated position information storing means to ensure that said imagelocation information is extracted.
 6. A camera calibrating apparatus tobe operative in combination with an imaging device which includes ahousing unit and an optical section supported by said housing unit toobtain image information through said optical section, and adapted tocalibrate optical position information indicative of a position of saidoptical section, comprising: first housing position information storingmeans for storing first housing position information indicative of aposition of said housing unit in the first coordinate system in which arevising marker is located; second housing position information storingmeans for storing second housing position information indicative of aposition of said housing unit in the second coordinate system; firstoptical position information calculating means for calculating firstoptical position information indicative of a position of said opticalsection in said first coordinate system on the basis of said imageinformation obtained by said imaging device and indicative of saidrevising marker; first optical position information storing means forstoring said first optical position information produced by said firstoptical position information producing means; second optical positioninformation calculating means for calculating second optical positioninformation indicative of a position of said optical section to saidsecond coordinate system from said second housing position informationstored by said second housing position information storing means on thebasis of said first housing position information stored by said firsthousing position information storing means and Said first opticalposition information stored by said first optical position informationstoring means; second optical position information storing means forstoring said second optical position information produced by said secondoptical position information producing means; and calibrating means forcalibrating said second optical position information stored by saidsecond optical position information storing means on the basis of amotion vector of said image information obtained by said imaging devicein said second coordinate system, wherein said calibrating meansincludes: plane-projected image producing means for producing aplane-projected image from said image information obtained by saidimaging device in said second coordinate system: plane-projected imagedividing means for dividing said plane-projected image produced by saidplane-projected image producing means into a plurality of imagesegments; motion vector extracting means for extracting said motionvector from said image segments divided by said plane-projected imagedividing means; calibration value calculating means for calculating acalibration value of said second optical position information stored bysaid second optical position information storing means on the basis ofsaid motion vector extracted by said motion vector extracting means; andoptical position information calibrating means for calibrating saidsecond optical position information stored by said second opticalposition information storing means on the basis of said calibrationvalue calculated by said calibration value calculating means.
 7. Acamera calibrating apparatus as set forth in claim 6, in which adividing marker is located in said second coordinate system in apredetermined relationship with said position of said housing unitrepresented by said second position information stored by said secondposition information storing means, and in which plane-projected imagedividing means is adapted to divide said plane-projected image producedby said plane-projected image producing means into a plurality of imagesegments on the basis of said image information of said dividing markerobtained by said imaging device.
 8. A camera calibrating apparatus to beoperative in combination with an imaging device which includes a housingunit and an optical section supported by said housing unit to obtainimage information through said optical section, and adapted to calibrateposition information of said optical section, comprising: first housingposition information storing means for storing first housing positioninformation indicative of a position of said housing unit in the firstcoordinate system in which a revising marker is located; second housingposition information storing means for storing second housing positioninformation indicative of a position of said housing unit in the secondcoordinate system in which an automotive vehicle is located; firstoptical position information producing calculating means for calculatingfirst optical position information indicative of a position of saidoptical section in said first coordinate system on the basis of saidimage information of said revising marker obtained by said imagingdevice; first optical position information storing means for storingsaid first optical position information produced by said first opticalposition information producing means; second optical positioninformation calculating means for calculating second optical positioninformation indicative of a position of said optical section to saidsecond coordinate system from said second housing position informationstored by said second housing position information storing means on thebasis of said first housing position information stored by said firsthousing position information storing means and said first opticalposition information stored by said first optical position informationstoring means; estimated position information calculating means forcalculating estimated position information indicative of a position ofsaid automotive vehicle to an image coordinate system of said imagingdevice on the basis of said second optical position information producedby said second optical position information producing means; secondoptical position information storing means for storing said secondoptical position information produced by said second optical positioninformation producing means; estimated position information storingmeans for storing said estimated position information produced by saidestimated position information estimating means; and calibrating meansfor calibrating said second optical position information stored by saidsecond optical position information storing means on the basis of saidimage information of said automotive vehicle obtained by said imagingdevice and said estimated position information stored by said estimatedposition information storing means.
 9. A camera calibrating apparatus asset forth in claim 8, in which said calibrating means includes: imagelocation information extracting means for extracting an image locationinformation indicative of an image location of said automotive vehiclein said image coordinate system of said imaging device on the basis ofsaid image information of said automotive vehicle obtained by saidimaging device; calibration value calculating means for calculating acalibration value of said second optical position information stored bysaid second optical position information storing means on the basis ofsaid image location information extracted by said image locationinformation extracting means and said estimated position informationstored by said estimated position information storing means; and opticalposition information calibrating means for calibrating said secondoptical position information stored by said second optical positioninformation storing means on the basis of said calibration valuecalculated by said calibration value calculating means.
 10. A cameracalibrating apparatus as set forth in claim 9, in which said calibrationvalue calculating means includes: superimposing means for superimposinga profile line of said automotive vehicle represented by said imagelocation information on a profile line of said automotive vehiclerepresented by said estimated position information; extracting means forextracting a plurality of points from said overlapped profile lines ofsaid automotive vehicle superimposed by said superimposing means, andcalculating means for calculating a calibration value of said secondoptical position information by comparing said points of said imagelocation information with said points of said estimated positioninformation.
 11. A camera calibrating apparatus as set forth in any oneof claims 1 to 6 and 7 to 10, in which said imaging device is mounted onan automotive vehicle.
 12. An imaging system comprises a cameracalibrating apparatus as set forth in any one of claims 1 to 6 and 7 to10.
 13. An imaging control system comprises a camera calibratingapparatus as set forth in any one of claims 1 to 6 and 7 to
 10. 14. Acamera calibrating method of calibrating optical position informationindicative of a position of an optical section supported by a housingunit of a camera for obtaining image information through said opticalsection, comprising: a first housing position information storing stepof storing first housing position information indicative of a positionof said housing unit in the first coordinate system in which a revisingmarker is located; a second housing position information storing step ofstoring second housing position information indicative of a position ofsaid housing unit in the second coordinate system in which a calibratingmarker is located; a first optical position information calculating stepof calculating first optical position information indicative of aposition of said optical section in said first coordinate system on thebasis of said image information of said revising marker obtained by saidimaging device; a first optical position information storing step ofstoring said first optical position information produced in said firstoptical position information producing step; a second optical positioninformation calculating step of calculating second optical positioninformation indicative of a position of said optical section to saidsecond coordinate system from said second housing position informationstored in said second housing position information storing step on thebasis of said first housing position information stored in said firsthousing position information storing step and said first opticalposition information stored in said first optical position informationstoring step; an estimated position information calculating step ofcalculating estimated position information indicative of a position ofsaid calibrating marker to an image coordinate system of said imagingdevice on the basis of said second optical position information producedin said second optical position information producing step; a secondoptical position information storing step of storing said second opticalposition information produced in said second optical positioninformation producing step; an estimated position information storingstep of storing said estimated position information produced in saidestimated position information estimating step; and a calibrating stepof calibrating said second optical position information stored in saidsecond optical position information storing step on the basis of saidimage information of said calibrating marker obtained by said imagingdevice and said estimated position information stored in said positioninformation storing step.
 15. A camera calibrating method of calibratingoptical position information indicative of a position of an opticalsection supported by a housing unit of a camera for obtaining imageinformation through said optical section, comprising: a first housingposition information storing step of storing first housing positioninformation indicative of a position of said housing unit in the firstcoordinate system in which a revising marker is located; a secondhousing position information storing step of storing second housingposition information indicative of a position of said housing unit inthe second coordinate system; a first optical position informationcalculating step of calculating first optical position informationindicative of a position of said optical section in said firstcoordinate system on the basis of said image information obtained bysaid imaging device and indicative of said revising marker; a firstoptical position information storing step of storing said first opticalposition information produced in said first optical position informationproducing step; a second optical position information calculating stepof calculating second optical position information indicative of aposition of said optical section to said second coordinate system fromsaid second housing position information stored in said second housingposition information storing step on the basis of said first housingposition information stored in said first housing position informationstoring step and said first optical position information stored in saidfirst optical position information storing step; a second opticalposition information storing step of storing said second opticalposition information produced in said second optical positioninformation producing step; and a calibrating step of calibrating saidsecond optical position information stored in said second opticalposition information storing step on the basis of a motion vector ofsaid image information obtained by said imaging device in said secondcoordinate system, wherein said calibrating step includes: aplane-projected image producing step of producing a plane-projectedimage from said image information obtained by said imaging device insaid second coordinate system; a plane-projected image dividing step ofdividing said plane-projected image produced by said plane-projectedimage producing step into a plurality of image segments; a motion vectorextracting step of extracting said motion vector from said imagesegments divided by said plane-projected image dividing step; acalibration value calculating step of calculating a calibration value ofsaid second optical position information stored by said second opticalposition information storing step on the basis of said motion vectorextracted by said motion vector extracting step; and a optical positioninformation calibrating step of calibrating said second optical positioninformation stored by said second optical position information storingstep on the basis of said calibration value calculated by saidcalibration value calculating step.
 16. A camera calibrating method ofcalibrating optical position information indicative of a position of anoptical section supported by a housing unit of a camera for obtainingimage information through said optical section, comprising: a firsthousing position information storing step of storing first housingposition information indicative of a position of said housing unit inthe first coordinate system in which a revising marker is located; asecond housing position information storing step of storing secondhousing position information indicative of a position of said housingunit in the second coordinate system in which an automotive vehicle islocated; a first optical position information calculating step ofcalculating first optical position information indicative of a positionof said optical section in said first coordinate system on the basis ofsaid image information of said revising marker obtained by said imagingdevice; a first optical position information storing step of storingsaid first optical position information produced in said first opticalposition information producing step; a second optical positioninformation calculating step of calculating second optical positioninformation indicative of a position of said optical section to saidsecond coordinate system from said second housing position informationstored in said second housing position information storing step on thebasis of said first housing position information stored in said firsthousing position information storing step and said first opticalposition information stored in said first optical position informationstoring step; an estimated position information calculating step ofcalculating estimated position information indicative of a position ofsaid automotive vehicle to an image coordinate system of said imagingdevice on the basis of said second optical position information producedin said second optical position information producing step; a secondoptical position information storing step of storing said second opticalposition information produced in said second optical positioninformation producing step; an estimated position information storingstep of storing said estimated position information produced in saidestimated position information estimating step; and a calibrating stepof calibrating said second optical position information stored in saidsecond optical position information storing step on the basis of saidimage information of said automotive vehicle obtained by said imagingdevice and said estimated position information stored in said estimatedposition information storing step.
 17. A camera calibrating method asset forth in any one of claims 14 to 16, in which said imaging device ismounted on an automotive vehicle.